Table maintenance for hmvp candidate storage

ABSTRACT

The disclosure relates to table maintenance for HMVP candidate storage. A method for processing video, including: determining, during a conversion between a first block of visual media data and a corresponding coded representation of the visual media data, the first block being coded with geometry partition mode; determining, based on at least one table storing history based motion vector prediction (HMVP) candidates which include motion information based on previously coded blocks, motion information of at least one sub-portion of the first block; performing the conversion of the first block using the determined motion information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/019,581, filed on Sep. 14, 2020, which is a continuation ofInternational Patent Application No. PCT/CN2019/115453, filed on Nov. 4,2019, which claims the priority to and benefits of International PatentApplication No. PCT/CN2018/113716, filed on Nov. 2, 2018 andInternational Patent Application No. PCT/CN2019/086174, filed on May 9,2019. All the aforementioned patent applications are hereby incorporatedby reference in their entireties.

TECHNICAL FIELD

This document is related to video and image coding and decodingtechnologies.

BACKGROUND

Digital video accounts for the largest bandwidth use on the internet andother digital communication networks. As the number of connected userdevices capable of receiving and displaying video increases, it isexpected that the bandwidth demand for digital video usage will continueto grow.

SUMMARY

The disclosed techniques may be used by video or image decoder orencoder embodiments for in which geometry partitions with history basedmotion vector prediction (HMVP) is used.

In one example aspect, a method of processing video is disclosed. Themethod includes performing a determination, by a processor, that a firstvideo block is intra-coded or non-merge inter-coded; determining, by theprocessor, a first sub-portion and a second sub-portion for the firstvideo block based on the determination that the first video block isintra-coded or non-merge inter-coded, wherein one or both of the firstsub-portion or the second sub-portion are non-rectangular and non-squareportions of the first video block; and performing further processing ofthe first video block using the first sub-portion and the secondsub-portion.

In another example aspect, a method of processing video includesperforming a determination, by a processor, that a first video block isintra-coded or non-merge inter-coded; determining, by the processor, afirst sub-portion and a second sub-portion for the first video block,wherein one or both of the first sub-portion or the second sub-portionare non-rectangular and non-square portions of the first video block;and performing further processing of the first video block using thefirst sub-portion and the second sub-portion, wherein at least onesub-portion is merge or non-merge inter coded and using a current imageas a reference image.

In another example aspect, a method of processing video includesperforming a determination, by a processor, that a first video block isintra-coded or non-merge inter-coded; determining, by the processor, afirst sub-portion and a second sub-portion for the first video block,wherein one or both of the first sub-portion or the second sub-portionare non-rectangular and non-square portions of the first video block;and performing further processing of the first video block using thefirst sub-portion and the second sub-portion, wherein performing furtherprocessing of the first video block using the first sub-portion and thesecond sub-portion is based on inter or intra coded information ofnon-adjacent spatial video blocks in relation to the first video block.

In another example aspect, a method of processing video includesperforming a determination that a first video block is coded withtriangular portion mode (TPM) using a triangular prediction portion ofthe first video block and that a second video block is coded usingnon-TPM using a non-triangular prediction portion of the second videoblock; performing further processing of the first video block and thesecond video block using stored HMVP candidates and storing HMVPcandidates associated with the first video block and the second videoblock.

In another example aspect, a method of processing video includesperforming a determination, by a processor, that a first video blockincludes prediction portions that are non-rectangular and non-squareportions of the first video block; identifying an HMVP candidate; addingone or more motion candidates derived from the HMVP candidate to a mergelist associated with video blocks that include prediction portions thatare non-rectangular and non-square; and performing further processing ofthe first video block using the merge list.

In another example aspect, a method for processing video includes:determining, during a conversion between a current block of visual mediadata and a corresponding coded representation of the visual media data,multiple sub-portions for the current block; determining intraprediction information of the multiple sub-portions; performing theconversion of the current block using the intra prediction informationof the multiple sub-portions; and wherein the current block isintra-coded, and at least one of the multiple sub-portions is anon-rectangular and non-square sub-portion.

In another example aspect, a method for processing video includes:determining, during a conversion between a current block of visual mediadata and a corresponding coded representation of the visual media data,multiple sub-portions for the current block; determining motioninformation of the multiple sub-portions; performing the conversion ofthe current block using the motion information of the multiplesub-portions; and wherein the current block is non-merge inter-coded,and at least one of the multiple sub-portions is a non-rectangular andnon-square sub-portion.

In another example aspect, a method of video processing includes:performing a conversion between a current block of visual media data anda corresponding coded representation of the visual media data, whereinthe current block is partitioned into multiple sub-portions according toa splitting pattern in which a first sub-portion has a non-rectangular,non-square shape; processing a first sub-portion with intra coding mode;and processing a second sub-portion with inter coding mode.

In another example aspect, a method of video processing includes:performing a conversion between a current block of visual media data anda corresponding coded representation of the visual media data, whereinthe current block is partitioned into multiple sub-portions according toa splitting pattern in which a first sub-portion has a non-rectangular,non-square shape; wherein the at least one of the multiple sub-portionsis merge or non-merge inter coded and uses a current picture as areference picture.

In another example aspect, a method of video processing includes:performing a conversion between a current block of visual media data anda corresponding coded representation of the visual media data, whereinthe current block is partitioned into multiple sub-portions according toa splitting pattern in which a first prediction partition has anon-rectangular, non-square shape; and performing the conversion usinginter or intra coded information of one or more non-adjacent spatialblocks.

In another example aspect, a method for processing video includes:determining, during a conversion between a first block of visual mediadata and a corresponding coded representation of the visual media data,the first block being coded with geometry partition mode; determining,based on at least one table storing history based motion vectorprediction (HMVP) candidates which include motion information based onpreviously coded blocks, motion information of at least one sub-portionof the first block; performing the conversion of the first block usingthe determined motion information.

In another example aspect, a method for processing video includes:determining, during a conversion between a first block of visual mediadata and a corresponding coded representation of the visual media data,the first block being coded with geometry partition mode; determiningmotion information of at least one sub-portion of the first block;performing the conversion of the first block using the motioninformation of the at least one sub-portion; wherein determining motioninformation of at least one sub-portion comprises using at least onehistory based motion vector prediction (HMVP) candidate which includesmotion information based on a previously coded block to construct amotion candidate list and determining the motion information from themotion candidate list.

In another example aspect, the above-described method may be implementedby a video encoder apparatus that comprises a processor.

In another example aspect, the above-described method may be implementedby a video decoder apparatus that comprises a processor.

In yet another example aspect, these methods may be embodied in the formof processor-executable instructions and stored on a computer-readableprogram medium.

These, and other, aspects are further described in the present document.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of a derivation process for merge candidateslist construction.

FIG. 2 shows an example of positions of spatial merge candidates.

FIG. 3 shows an example of candidate pairs considered for redundancycheck of spatial merge candidates.

FIG. 4 shows an example of positions for the second PU of N×2N and 2N×Npartitions.

FIG. 5 shows an example of motion vector scaling for a temporal mergecandidate.

FIG. 6 shows an example of candidate positions for temporal mergecandidate, C0 and C1.

FIG. 7 shows an example of a combined bi-predictive merge candidate.

FIG. 8 shows an example of a derivation process for motion vectorprediction candidates.

FIG. 9 shows an example of motion vector scaling for a spatial motionvector candidate.

FIG. 10 shows an example of ATMVP motion prediction for a CU.

FIG. 11 shows an example of one CU with four sub-blocks (A-D) and itsneighbouring blocks (a-d).

FIG. 12 shows an example of a flowchart of encoding with different MVprecision.

FIG. 13 shows examples of splitting a CU into two triangular predictionunits (two partition types).

FIG. 14 shows an example of a position of neighboring blocks.

FIG. 15 shows an example of a CU applying the 1st weighting factorgroup.

FIG. 16 shows an example of motion vector storage.

FIG. 17 shows an example of neighboring blocks (A and L) used forcontext selection in TPM flag coding.

FIG. 18 shows an example of sub-blocks where OBMC applies.

FIG. 19 shows an example of neighboring samples used for deriving ICparameters.

FIG. 20 shows an example of a simplified affine motion model.

FIG. 21 shows an example of affine MVF per sub-block.

FIG. 22 shows an example of 4-paramenter affine model (a) and6-parameter affine model (b).

FIG. 23 shows an example of MVP for AF_INTER.

FIG. 24 shows an example of candidates for AF_MERGE.

FIG. 25 shows an example of candidates position for affine merge mode.

FIG. 26 shows an example of optical flow trajectory.

FIG. 27 shows an example of BIO w/o block extension: a) access positionsoutside of the block; b) padding is used in order to avoid extra memoryaccess and calculation.

FIG. 28 shows an example of DMVR based on bilateral template matching.

FIG. 29 is a block diagram of an example of a video processingapparatus.

FIG. 30 shows a block diagram of an example implementation of a videoencoder.

FIG. 31 is a flowchart for an example of a video processing method.

FIG. 32 is a flowchart for an example of a video processing method.

FIG. 33 is a flowchart for an example of a video processing method.

FIG. 34 is a flowchart for an example of a video processing method.

FIG. 35 is a flowchart for an example of a video processing method.

FIG. 36 is a flowchart for an example of a video processing method.

FIG. 37 is a flowchart for an example of a video processing method.

FIG. 38 is a flowchart for an example of a video processing method.

FIG. 39 is a flowchart for an example of a video processing method.

FIG. 40 is a flowchart for an example of a video processing method.

FIG. 41 is a flowchart for an example of a video processing method.

FIG. 42 is a flowchart for an example of a video processing method.

DETAILED DESCRIPTION

The present document provides various techniques that can be used by adecoder of image or video bitstreams to improve the quality ofdecompressed or decoded digital video or images. For brevity, the term“video” is used herein to include both a sequence of pictures(traditionally called video) and individual images. Furthermore, a videoencoder may also implement these techniques during the process ofencoding in order to reconstruct decoded frames used for furtherencoding.

Section headings are used in the present document for ease ofunderstanding and do not limit the embodiments and techniques to thecorresponding sections. As such, embodiments from one section can becombined with embodiments from other sections.

1. Summary

This patent document is related to video coding technologies.Specifically, it is related to motion vector coding under geometrypartition in video coding. It may be applied to the existing videocoding standard like HEVC, or the standard (Versatile Video Coding) tobe finalized. It may be also applicable to future video coding standardsor video codec.

2. Background

Video coding standards have evolved primarily through the development ofthe well-known ITU-T and ISO/IEC standards. The ITU-T produced H.261 andH.263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the twoorganizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4Advanced Video Coding (AVC) and H.265/HEVC standards. Since H.262, thevideo coding standards are based on the hybrid video coding structurewherein temporal prediction plus transform coding are utilized. Toexplore the future video coding technologies beyond HEVC, Joint VideoExploration Team (JVET) was founded by VCEG and MPEG jointly in 2015.Since then, many new methods have been adopted by JVET and put into thereference software named Joint Exploration Model (JEM. In April 2018,the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1SC29/WG11 (MPEG) was created to work on the VVC standard targeting at50% bitrate reduction compared to HEVC.

FIG. 30 is a block diagram of an example implementation of a videoencoder. FIG. 30 shows that the encoder implementation has a feedbackpath built in in which the video encoder also performs video decodingfunctionality (reconstructing compressed representation of video datafor use in encoding of next video data).

2.1 Inter Prediction in HEVC/H.265

Each inter-predicted PU has motion parameters for one or two referencepicture lists. Motion parameters include a motion vector and a referencepicture index. Usage of one of the two reference picture lists may alsobe signalled using inter_pred_idc. Motion vectors may be explicitlycoded as deltas relative to predictors.

When a CU is coded with skip mode, one PU is associated with the CU, andthere are no significant residual coefficients, no coded motion vectordelta or reference picture index. A merge mode is specified whereby themotion parameters for the current PU are obtained from neighbouring PUs,including spatial and temporal candidates. The merge mode can be appliedto any inter-predicted PU, not only for skip mode. The alternative tomerge mode is the explicit transmission of motion parameters, wheremotion vector (to be more precise, motion vector difference compared toa motion vector predictor), corresponding reference picture index foreach reference picture list and reference picture list usage aresignalled explicitly per each PU. Such a mode is named Advanced motionvector prediction (AMVP) in this disclosure.

When signalling indicates that one of the two reference picture lists isto be used, the PU is produced from one block of samples. This isreferred to as ‘uni-prediction’. Uni-prediction is available both forP-slices and B-slices.

When signalling indicates that both of the reference picture lists areto be used, the PU is produced from two blocks of samples. This isreferred to as ‘bi-prediction’. Bi-prediction is available for B-slicesonly.

The following text provides the details on the inter prediction modesspecified in HEVC. The description will start with the merge mode.

2.1.1. Reference Picture List

In HEVC, the term inter prediction is used to denote prediction derivedfrom data elements (e.g., sample values or motion vectors) of referencepictures other than the current decoded picture. Like in H.264/AVC, apicture can be predicted from multiple reference pictures. The referencepictures that are used for inter prediction are organized in one or morereference picture lists. The reference index identifies which of thereference pictures in the list should be used for creating theprediction signal.

A single reference picture list, List 0, is used for a P slice and tworeference picture lists, List 0 and List 1 are used for B slices. Itshould be noted reference pictures included in List 0/1 could be frompast and future pictures in terms of capturing/display order.

2.1.2 Merge Mode

2.1.2.1 Derivation of Candidates for Merge Mode

When a PU is predicted using merge mode, an index pointing to an entryin the merge candidates list is parsed from the bitstream and used toretrieve the motion information. The construction of this list isspecified in the HEVC standard and can be summarized according to thefollowing sequence of steps:

Step 1: Initial candidates derivation

Step 1.1: Spatial candidates derivation

Step 1.2: Redundancy check for spatial candidates

Step 1.3: Temporal candidates derivation

Step 2: Additional candidates insertion

Step 2.1: Creation of bi-predictive candidates

Step 2.2: Insertion of zero motion candidates

These steps are also schematically depicted in FIG. 1. For spatial mergecandidate derivation, a maximum of four merge candidates are selectedamong candidates that are located in five different positions. Fortemporal merge candidate derivation, a maximum of one merge candidate isselected among two candidates. Since constant number of candidates foreach PU is assumed at decoder, additional candidates are generated whenthe number of candidates obtained from step 1 does not reach the maximumnumber of merge candidate (MaxNumMergeCand) which is signalled in sliceheader. Since the number of candidates is constant, index of best mergecandidate is encoded using truncated unary binarization (TU). If thesize of CU is equal to 8, all the PUs of the current CU share a singlemerge candidate list, which is identical to the merge candidate list ofthe 2N×2N prediction unit.

In the following, the operations associated with the aforementionedsteps are detailed.

FIG. 1 shows an example of a derivation process for merge candidateslist construction.

2.1.2.2. Spatial Candidates Derivation

In the derivation of spatial merge candidates, a maximum of four mergecandidates are selected among candidates located in the positionsdepicted in FIG. 2. The order of derivation is A₁, B₁, B₀, A₀ and B₂.Position B₂ is considered only when any PU of position A₁, B₁, B₀, A₀ isnot available (e.g. because it belongs to another slice or tile) or isintra coded. After candidate at position A₁ is added, the addition ofthe remaining candidates is subject to a redundancy check which ensuresthat candidates with same motion information are excluded from the listso that coding efficiency is improved. To reduce computationalcomplexity, not all possible candidate pairs are considered in thementioned redundancy check. Instead only the pairs linked with an arrowin FIG. 3 are considered and a candidate is only added to the list ifthe corresponding candidate used for redundancy check has not the samemotion information. Another source of duplicate motion information isthe “second PU” associated with partitions different from 2N×2N. As anexample, FIG. 4 depicts the second PU for the case of N×2N and 2N×N,respectively. When the current PU is partitioned as N×2N, candidate atposition A₁ is not considered for list construction. In fact, by addingthis candidate will lead to two prediction units having the same motioninformation, which is redundant to just have one PU in a coding unit.Similarly, position B₁ is not considered when the current PU ispartitioned as 2N×N.

FIG. 2 shows an example of positions of spatial merge candidates.

FIG. 3 shows an example of candidate pairs considered for redundancycheck of spatial merge candidates.

FIG. 4 shows an example of positions for the second PU of N×2N and 2N×Npartitions.

2.1.2.3. Temporal Candidates Derivations

In this step, only one candidate is added to the list. Particularly, inthe derivation of this temporal merge candidate, a scaled motion vectoris derived based on co-located PU belonging to the picture which has thesmallest POC difference with current picture within the given referencepicture list. The reference picture list to be used for derivation ofthe co-located PU is explicitly signalled in the slice header. Thescaled motion vector for temporal merge candidate is obtained asillustrated by the dotted line in FIG. 5, which is scaled from themotion vector of the co-located PU using the POC distances, tb and td,where tb is defined to be the POC difference between the referencepicture of the current picture and the current picture and td is definedto be the POC difference between the reference picture of the co-locatedpicture and the co-located picture. The reference picture index oftemporal merge candidate is set equal to zero. A practical realizationof the scaling process is described in the HEVC specification. For aB-slice, two motion vectors, one is for reference picture list 0 and theother is for reference picture list 1, are obtained and combined to makethe bi-predictive merge candidate.

FIG. 5 shows an example of motion vector scaling for temporal mergecandidate.

In the co-located PU (Y) belonging to the reference frame, the positionfor the temporal candidate is selected between candidates C₀ and C₁, asdepicted in FIG. 6. If PU at position C₀ is not available, is intracoded, or is outside of the current CTU row, position C₁ is used.Otherwise, position C₀ is used in the derivation of the temporal mergecandidate.

FIG. 6 shows an example of candidate positions for temporal mergecandidate, C0 and C1.

2.1.2.4. Additional Candidates Insertion

Besides spatial and temporal merge candidates, there are two additionaltypes of merge candidates: combined bi-predictive merge candidate andzero merge candidate. Combined bi-predictive merge candidates aregenerated by utilizing spatial and temporal merge candidates. Combinedbi-predictive merge candidate is used for B-Slice only. The combinedbi-predictive candidates are generated by combining the first referencepicture list motion parameters of an initial candidate with the secondreference picture list motion parameters of another. If these two tuplesprovide different motion hypotheses, they will form a new bi-predictivecandidate. As an example, FIG. 7 depicts the case when two candidates inthe original list (on the left), which have mvL0 and refIdxL0 or mvL1and refldxL1, are used to create a combined bi-predictive mergecandidate added to the final list (on the right). There are numerousrules regarding the combinations which are considered to generate theseadditional merge candidates.

FIG. 7 shows an example of combined bi-predictive merge candidate.

Zero motion candidates are inserted to fill the remaining entries in themerge candidates list and therefore hit the MaxNumMergeCand capacity.These candidates have zero spatial displacement and a reference pictureindex which starts from zero and increases every time a new zero motioncandidate is added to the list. The number of reference frames used bythese candidates is one and two for uni and bi-directional prediction,respectively. Finally, no redundancy check is performed on thesecandidates.

2.1.2.5. Motion Estimation Regions for Parallel Processing

To speed up the encoding process, motion estimation can be performed inparallel whereby the motion vectors for all prediction units inside agiven region are derived simultaneously. The derivation of mergecandidates from spatial neighbourhood may interfere with parallelprocessing as one prediction unit cannot derive the motion parametersfrom an adjacent PU until its associated motion estimation is completed.To mitigate the trade-off between coding efficiency and processinglatency, HEVC defines the motion estimation region (MER) whose size issignalled in the picture parameter set using the “log2_parallel_merge_level_minus2” syntax element. When a MER is defined,merge candidates falling in the same region are marked as unavailableand therefore not considered in the list construction.

2.1.3. AMVP

AMVP exploits spatio-temporal correlation of motion vector withneighbouring PUs, which is used for explicit transmission of motionparameters. For each reference picture list, a motion vector candidatelist is constructed by firstly checking availability of left, abovetemporally neighbouring PU positions, removing redundant candidates andadding zero vector to make the candidate list to be constant length.Then, the encoder can select the best predictor from the candidate listand transmit the corresponding index indicating the chosen candidate.Similarly with merge index signalling, the index of the best motionvector candidate is encoded using truncated unary. The maximum value tobe encoded in this case is 2 (see FIG. 8). In the following sections,details about derivation process of motion vector prediction candidateare provided.

2.1.3.1 Derivation of AMVP Candidates

FIG. 8 summarizes derivation process for motion vector predictioncandidate.

In motion vector prediction, two types of motion vector candidates areconsidered: spatial motion vector candidate and temporal motion vectorcandidate. For spatial motion vector candidate derivation, two motionvector candidates are eventually derived based on motion vectors of eachPU located in five different positions as depicted in FIG. 2.

For temporal motion vector candidate derivation, one motion vectorcandidate is selected from two candidates, which are derived based ontwo different co-located positions. After the first list ofspatio-temporal candidates is made, duplicated motion vector candidatesin the list are removed. If the number of potential candidates is largerthan two, motion vector candidates whose reference picture index withinthe associated reference picture list is larger than 1 are removed fromthe list. If the number of spatio-temporal motion vector candidates issmaller than two, additional zero motion vector candidates is added tothe list.

2.1.3.2. Spatial Motion Vector Candidates

In the derivation of spatial motion vector candidates, a maximum of twocandidates are considered among five potential candidates, which arederived from PUs located in positions as depicted in FIG. 2, thosepositions being the same as those of motion merge. The order ofderivation for the left side of the current PU is defined as A0, A1, andscaled A0, scaled A1. The order of derivation for the above side of thecurrent PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2.For each side there are therefore four cases that can be used as motionvector candidate, with two cases not required to use spatial scaling,and two cases where spatial scaling is used. The four different casesare summarized as follows.

No Spatial Scaling

(1) Same reference picture list, and same reference picture index (samePOC)

(2) Different reference picture list, but same reference picture (samePOC)

Spatial Scaling

(3) Same reference picture list, but different reference picture(different POC)

(4) Different reference picture list, and different reference picture(different POC)

The no-spatial-scaling cases are checked first followed by the spatialscaling. Spatial scaling is considered when the POC is different betweenthe reference picture of the neighbouring PU and that of the current PUregardless of reference picture list. If all PUs of left candidates arenot available or are intra coded, scaling for the above motion vector isallowed to help parallel derivation of left and above MV candidates.Otherwise, spatial scaling is not allowed for the above motion vector.

FIG. 9 shows an example of motion vector scaling for spatial motionvector candidate.

In a spatial scaling process, the motion vector of the neighbouring PUis scaled in a similar manner as for temporal scaling, as depicted asFIG. 9. The main difference is that the reference picture list and indexof current PU is given as input; the actual scaling process is the sameas that of temporal scaling.

2.1.3.3. Temporal Motion Vector Candidates

Apart for the reference picture index derivation, all processes for thederivation of temporal merge candidates are the same as for thederivation of spatial motion vector candidates (see FIG. 6). Thereference picture index is signalled to the decoder.

2.2. New Inter Prediction Methods in JEM

2.2.1. Sub-CU Based Motion Vector Prediction

In the JEM with QTBT, each CU can have at most one set of motionparameters for each prediction direction. Two sub-CU level motion vectorprediction methods are considered in the encoder by splitting a large CUinto sub-CUs and deriving motion information for all the sub-CUs of thelarge CU. Alternative temporal motion vector prediction (ATMVP) methodallows each CU to fetch multiple sets of motion information frommultiple blocks smaller than the current CU in the collocated referencepicture. In spatial-temporal motion vector prediction (STMVP) methodmotion vectors of the sub-CUs are derived recursively by using thetemporal motion vector predictor and spatial neighbouring motion vector.

To preserve more accurate motion field for sub-CU motion prediction, themotion compression for the reference frames is currently disabled.

FIG. 10 shows an example ATMVP motion prediction for a CU.

2.2.1.1. Alternative Temporal Motion Vector Prediction

In the alternative temporal motion vector prediction (ATMVP) method, themotion vectors temporal motion vector prediction (TMVP) is modified byfetching multiple sets of motion information (including motion vectorsand reference indices) from blocks smaller than the current CU. Thesub-CUs are square N×N blocks (N is set to 4 by default).

ATMVP predicts the motion vectors of the sub-CUs within a CU in twosteps. The first step is to identify the corresponding block in areference picture with a so-called temporal vector. The referencepicture is called the motion source picture. The second step is to splitthe current CU into sub-CUs and obtain the motion vectors as well as thereference indices of each sub-CU from the block corresponding to eachsub-CU.

In the first step, a reference picture and the corresponding block isdetermined by the motion information of the spatial neighbouring blocksof the current CU. To avoid the repetitive scanning process ofneighbouring blocks, the first merge candidate in the merge candidatelist of the current CU is used. The first available motion vector aswell as its associated reference index are set to be the temporal vectorand the index to the motion source picture. This way, in ATMVP, thecorresponding block may be more accurately identified, compared withTMVP, wherein the corresponding block (sometimes called collocatedblock) is always in a bottom-right or center position relative to thecurrent CU.

In the second step, a corresponding block of the sub-CU is identified bythe temporal vector in the motion source picture, by adding to thecoordinate of the current CU the temporal vector. For each sub-CU, themotion information of its corresponding block (the smallest motion gridthat covers the center sample) is used to derive the motion informationfor the sub-CU. After the motion information of a corresponding N×Nblock is identified, it is converted to the motion vectors and referenceindices of the current sub-CU, in the same way as TMVP of HEVC, whereinmotion scaling and other procedures apply. For example, the decoderchecks whether the low-delay condition (i.e. the POCs of all referencepictures of the current picture are smaller than the POC of the currentpicture) is fulfilled and possibly uses motion vector MV_(x) (the motionvector corresponding to reference picture list X) to predict motionvector MV_(y) (with X being equal to 0 or 1 and Y being equal to 1-X)for each sub-CU.

2.2.1.2. Spatial-Temporal Motion Vector Prediction (STMVP)

In this method, the motion vectors of the sub-CUs are derivedrecursively, following raster scan order. FIG. 11 illustrates thisconcept. Let us consider an 8×8 CU which contains four 4×4 sub-CUs A, B,C, and D. The neighbouring 4×4 blocks in the current frame are labelledas a, b, c, and d.

The motion derivation for sub-CU A starts by identifying its two spatialneighbours. The first neighbour is the N×N block above sub-CU A (blockc). If this block c is not available or is intra coded the other N×Nblocks above sub-CU A are checked (from left to right, starting at blockc). The second neighbour is a block to the left of the sub-CU A (blockb). If block b is not available or is intra coded other blocks to theleft of sub-CU A are checked (from top to bottom, staring at block b).The motion information obtained from the neighbouring blocks for eachlist is scaled to the first reference frame for a given list. Next,temporal motion vector predictor (TMVP) of sub-block A is derived byfollowing the same procedure of TMVP derivation as specified in HEVC.The motion information of the collocated block at location D is fetchedand scaled accordingly. Finally, after retrieving and scaling the motioninformation, all available motion vectors (up to 3) are averagedseparately for each reference list. The averaged motion vector isassigned as the motion vector of the current sub-CU.

2.2.1.3. Sub-CU Motion Prediction Mode Signaling

The sub-CU modes are enabled as additional merge candidates and there isno additional syntax element required to signal the modes. Twoadditional merge candidates are added to merge candidates list of eachCU to represent the ATMVP mode and STMVP mode. Up to seven mergecandidates are used, if the sequence parameter set indicates that ATMVPand STMVP are enabled. The encoding logic of the additional mergecandidates is the same as for the merge candidates in the HM, whichmeans, for each CU in P or B slice, two more RD checks is needed for thetwo additional merge candidates.

In the JEM, all bins of merge index is context coded by CABAC. While inHEVC, only the first bin is context coded and the remaining bins arecontext by-pass coded.

2.2.2. Adaptive Motion Vector Difference Resolution

In HEVC, motion vector differences (MVDs) (between the motion vector andpredicted motion vector of a PU) are signalled in units of quarter lumasamples when use_integer_mv_flag is equal to 0 in the slice header. Inthe JEM, a locally adaptive motion vector resolution (LAMVR) isintroduced. In the JEM, MVD can be coded in units of quarter lumasamples, integer luma samples or four luma samples. The MVD resolutionis controlled at the coding unit (CU) level, and MVD resolution flagsare conditionally signalled for each CU that has at least one non-zeroMVD components.

For a CU that has at least one non-zero MVD components, a first flag issignalled to indicate whether quarter luma sample MV precision is usedin the CU. When the first flag (equal to 1) indicates that quarter lumasample MV precision is not used, another flag is signalled to indicatewhether integer luma sample MV precision or four luma sample MVprecision is used.

When the first MVD resolution flag of a CU is zero, or not coded for aCU (meaning all MVDs in the CU are zero), the quarter luma sample MVresolution is used for the CU. When a CU uses integer-luma sample MVprecision or four-luma-sample MV precision, the MVPs in the AMVPcandidate list for the CU are rounded to the corresponding precision.

In the encoder, CU-level RD checks are used to determine which MVDresolution is to be used for a CU. That is, the CU-level RD check isperformed three times for each MVD resolution. To accelerate encoderspeed, the following encoding schemes are applied in the JEM.

During RD check of a CU with normal quarter luma sample MVD resolution,the motion information of the current CU (integer luma sample accuracy)is stored. The stored motion information (after rounding) is used as thestarting point for further small range motion vector refinement duringthe RD check for the same CU with integer luma sample and 4 luma sampleMVD resolution so that the time-consuming motion estimation process isnot duplicated three times.

RD check of a CU with 4 luma sample MVD resolution is conditionallyinvoked. For a CU, when RD cost integer luma sample MVD resolution ismuch larger than that of quarter luma sample MVD resolution, the RDcheck of 4 luma sample MVD resolution for the CU is skipped.

The encoding process is shown in FIG. 12. First, 1/4 pel MV is testedand the RD cost is calculated and denoted as RDCost0, then integer MV istested and the RD cost is denoted as RDCost1. If RDCost1<th*RDCost0(wherein th is a positive value), then 4-pel MV is tested; otherwise,4-pel MV is skipped. Basically, motion information and RD cost etc. arealready known for 1/4 pel MV when checking integer or 4-pel MV, whichcan be reused to speed up the encoding process of integer or 4-pel MV.

2.2.3. Triangular Prediction Mode

The concept of the triangular prediction mode (TPM) is to introduce anew triangular partition for motion compensated prediction. As shown inFIG. 13, it splits a CU into two triangular prediction units, in eitherdiagonal or inverse diagonal direction. Each triangular prediction unitin the CU is inter-predicted using its own uni-prediction motion vectorand reference frame index which are derived from a uni-predictioncandidate list. An adaptive weighting process is performed to thediagonal edge after predicting the triangular prediction units. Then,the transform and quantization process are applied to the whole CU. Itis noted that this mode is only applied to skip and merge modes.

2.2.3.1 Uni-Prediction Candidate List for TPM

The uni-prediction candidate list consists of five uni-prediction motionvector candidates. It is derived from seven neighboring blocks includingfive spatial neighboring blocks (1 to 5) and two temporal co-locatedblocks (6 to 7), as shown in FIG. 14. The motion vectors of the sevenneighboring blocks are collected and put into the uni-predictioncandidate list according in the order of uni-prediction motion vectors,L0 motion vector of bi-prediction motion vectors, L1 motion vector ofbi-prediction motion vectors, and averaged motion vector of the L0 andL1 motion vectors of bi-prediction motion vectors. If the number ofcandidates is less than five, zero motion vector is added to the list.

More specifically, the following steps are involved:

Obtain motion candidates from A1, B1, B0, A0, B2, Col and Col2(corresponding to block 1-7 in FIG. 14) without any pruning operations.

Set variable numCurrMergeCand=0

For each motion candidates derived from A1, B1, B0, A0, B2, Col and Col2and numCurrMergeCand is less than 5, if the motion candidate isuni-prediction (either from List 0 or List 1), it is added to the mergelist with numCurrMergeCand increased by 1. Such added motion candidatesis named ‘originally uni-predicted candidate’.

Full pruning is applied.

For each motion candidates derived from A1, B1, B0, A0, B2, Col and Col2and numCurrMergeCand is less than 5, if the motion candidate isbi-prediction, the motion information from List 0 is added to the mergelist (that is, modified to be uni-prediction from List 0) andnumCurrMergeCand increased by 1. Such added motion candidates is named‘Truncated List0-predicted candidate’.

Full pruning is applied.

For each motion candidates derived from A1, B1, B0, A0, B2, Col and Col2and numCurrMergeCand is less than 5, if the motion candidate isbi-prediction, the motion information from List 1 is added to the mergelist (that is, modified to be uni-prediction from List 1) andnumCurrMergeCand increased by 1. Such added motion candidates is named‘Truncated List1-predicted candidate’.

Full pruning is applied.

For each motion candidates derived from A1, B1, B0, A0, B2, Col and Col2and numCurrMergeCand is less than 5, if the motion candidate isbi-prediction,

If List 0 reference picture's slice QP is smaller than List 1 referencepicture's slice QP, the motion information of List 1 is firstly scaledto List 0 reference picture, and the average of the two MVs (one is fromoriginal List 0, and the other is the scaled MV from List 1) is added tothe merge list, that is averaged uni-prediction from List 0 motioncandidate and numCurrMergeCand increased by 1.

Otherwise, the motion information of List 0 is firstly scaled to List 1reference picture, and the average of the two MVs (one is from originalList 1, and the other is the scaled MV from List 0) is added to themerge list, that is averaged uni-prediction from List 1 motion candidateand numCurrMergeCand increased by 1.

Full pruning is applied.

If numCurrMergeCand is less than 5, zero motion vector candidates areadded.

2.2.3.2. Adaptive Weighting Process

After predicting each triangular prediction unit, an adaptive weightingprocess is applied to the diagonal edge between the two triangularprediction units to derive the final prediction for the whole CU. Twoweighting factor groups are defined as follows:

1st weighting factor group: {7/8, 6/8, 4/8, 2/8, 1/8} and {7/8, 4/8,1/8} are used for the luminance and the chrominance samples,respectively;

2nd weighting factor group: {7/8, 6/8, 5/8, 4/8, 3/8, 2/8, 1/8} and{6/8, 4/8, 2/8} are used for the luminance and the chrominance samples,respectively.

Weighting factor group is selected based on the comparison of the motionvectors of two triangular prediction units. The 2nd weighting factorgroup is used when the reference pictures of the two triangularprediction units are different from each other or their motion vectordifference is larger than 16 pixels. Otherwise, the 1st weighting factorgroup is used. An example is shown in FIG. 15.

2.2.3.3. Motion Vector Storage

The motion vectors (Mv1 and Mv2 in FIG. 16) of the triangular predictionunits are stored in 4×4 grids. For each 4×4 grid, either uni-predictionor bi-prediction motion vector is stored depending on the position ofthe 4×4 grid in the CU. As shown in FIG. 16, uni-prediction motionvector, either Mv1 or Mv2, is stored for the 4×4 grid located in thenon-weighted area (that is, not located at the diagonal edge). On theother hand, a bi-prediction motion vector is stored for the 4×4 gridlocated in the weighted area. The bi-prediction motion vector is derivedfrom Mv1 and Mv2 according to the following rules:

In the case that Mv1 and Mv2 have motion vector from differentdirections (L0 or L1), Mv1 and Mv2 are simply combined to form thebi-prediction motion vector.

In the case that both Mv1 and Mv2 are from the same L0 (or L1)direction,

If the reference picture of Mv2 is the same as a picture in the L1 (orL0) reference picture list, Mv2 is scaled to the picture. Mv1 and thescaled Mv2 are combined to form the bi-prediction motion vector.

If the reference picture of Mv1 is the same as a picture in the L1 (orL0) reference picture list, Mv1 is scaled to the picture. The scaled Mv1and Mv2 are combined to form the bi-prediction motion vector.

Otherwise, only Mv1 is stored for the weighted area.

2.2.3.4. Signaling of Triangular Prediction Mode (TPM)

One bit flag to indicate whether TPM is used may be firstly signaled.Afterwards, the indications of two splitting patterns (as depicted inFIG. 13), and selected merge indices for each of the two partitions arefurther signaled.

2.2.3.4.1 Signaling of TPM Flag

Let's denote one luma block's width and height by W and H, respectively.If W*H<64, triangular prediction mode is disabled.

When one block is coded with affine mode, triangular prediction mode isalso disabled.

When one block is coded with merge mode, one bit flag may be signaled toindicate whether the triangular prediction mode is enabled or disabledfor the block.

The flag is coded with 3 contexts, based on the following equation:

Ctx index=((left block L available && L is coded with TPM?) 1:0)+((Aboveblock A available && A is coded with TPM?) 1:0);

2.2.3.4.2. Signaling of an Indication of Two Splitting Patterns (asDepicted in FIG. 13), and Selected Merge Indices for Each of the TwoPartitions

It is noted that splitting patterns, merge indices of two partitions arejointly coded. It is restricted that the two partitions couldn't use thesame reference index. Therefore, there are 2 (splitting patterns)*N(maximum number of merge candidates)*(N−1) possibilities wherein N isset to 5. One indication is coded and the mapping between the partitiontype, two merge indices are derived from the array defined below:

const uint8_t g_TriangleCombination[TRIANGLE_MAX_NUM_CANDS][3]={

{0, 1, 0}, {1, 0, 1}, {1, 0, 2}, {0, 0, 1}, {0, 2, 0},

{1, 0, 3}, {1, 0, 4}, {1, 1, 0}, {0, 3, 0}, {0, 4, 0},

{0, 0, 2}, {0, 1, 2}, {1, 1, 2}, {0, 0, 4}, {0, 0, 3},

{0, 1, 3}, {0, 1, 4}, {1, 1, 4}, {1, 1, 3}, {1, 2, 1},

{1, 2, 0}, {0, 2, 1}, {0, 4, 3}, {1, 3, 0}, {1, 3, 2},

{1, 3, 4}, {1, 4, 0}, {1, 3, 1}, {1, 2, 3}, {1, 4, 1},

{0, 4, 1}, {0, 2, 3}, {1, 4, 2}, {0, 3, 2}, {1, 4, 3},

{0, 3, 1}, {0, 2, 4}, {1, 2, 4}, {0, 4, 2}, {0, 3, 4}};

Partition type (45 degree or 135 degree)=g_TriangleCombination[signaledindication][0];

Merge index of candidate A=g_TriangleCombination[signaledindication][1];

Merge index of candidate B=g_TriangleCombination[signaledindication][2];

Once the two motion candidates A and B are derived, the two partitions'(PU1 and PU2) motion information could be set either from A or B.Whether PU1 uses the motion information of merge candidate A or B isdependent on the prediction directions of the two motion candidates.

Table 1 Table 1 shows the relationship between two derived motioncandidates A and B, with the two partitions.

TABLE 1 Derivation of partitions' motion information from derived twomerge candidates (A, B) Prediction Prediction PU1's motion PU2's motiondirection of A direction of B information information L0 L0 A (L0) B(L0) L1 L1 B (L1) A (L1) L0 L1 A (L0) B (L1) L1 L0 B (L0) A (L1)

2.2.3.4.3. Entropy Coding of the Indication (Denoted byMerge_Triangle_Idx)

merge_triangle_idx is within the range [0, 39], inclusively. K-th orderExponential Golomb (EG) code is used for binarization ofmerge_triangle_idx wherein K is set to 1.

K-th order EG

To encode larger numbers in fewer bits (at the expense of using morebits to encode smaller numbers), this can be generalized using anonnegative integer parameter k. To encode a nonnegative integer x in anorder-k exp-Golomb code:

Encode [x/2^(k)] using order-0 exp-Golomb code described above, then

Encode x mod 2^(k) in binary

TABLE 2 Exp-Golomb-k coding examples x k = 0 k = 1 k = 2 0 1   10 100 1010   11 101 2 011  0100 110 3 00100  0101 111 4 00101  0110 01000 500110  0111 01001 6 00111 001000  01010 7 0001000 001001  01011 80001001 001010  01100 9 0001010 001011  01101 10 0001011 001100w 0111011 0001100 001101  01111 12 0001101 001110  0010000 13 0001110 001111 0010001 14 0001111 00010000   0010010 15 000010000 00010001   0010011 16000010001 00010010   0010100 17 000010010 00010011   0010101 18000010011 00010100   0010110 19 000010100 00010101   0010111

2.2.4. Overlapped Block Motion Compensation

Overlapped Block Motion Compensation (OBMC) has previously been used inH.263. In the JEM, unlike in H.263, OBMC can be switched on and offusing syntax at the CU level. When OBMC is used in the JEM, the OBMC isperformed for all motion compensation (MC) block boundaries except theright and bottom boundaries of a CU. Moreover, it is applied for boththe luma and chroma components. In the JEM, a MC block is correspondingto a coding block. When a CU is coded with sub-CU mode (includes sub-CUmerge, affine and FRUC mode), each sub-block of the CU is a MC block. Toprocess CU boundaries in a uniform fashion, OBMC is performed atsub-block level for all MC block boundaries, where sub-block size is setequal to 4×4, as illustrated in FIG. 18.

When OBMC applies to the current sub-block, besides current motionvectors, motion vectors of four connected neighbouring sub-blocks, ifavailable and are not identical to the current motion vector, are alsoused to derive prediction block for the current sub-block. Thesemultiple prediction blocks based on multiple motion vectors are combinedto generate the final prediction signal of the current sub-block.

Prediction block based on motion vectors of a neighbouring sub-block isdenoted as P_(N), with N indicating an index for the neighbouring above,below, left and right sub-blocks and prediction block based on motionvectors of the current sub-block is denoted as P_(C). When P_(N) isbased on the motion information of a neighbouring sub-block thatcontains the same motion information to the current sub-block, the OBMCis not performed from P_(N). Otherwise, every sample of P_(N) is addedto the same sample in P_(c), i.e., four rows/columns of P_(N) are addedto P_(C). The weighting factors {1/4, 1/8, 1/16, 1/32} are used forP_(N) and the weighting factors {3/4, 7/8, 15/16, 31/32} are used forP_(C). The exception are small MC blocks, (i.e., when height or width ofthe coding block is equal to 4 or a CU is coded with sub-CU mode), forwhich only two rows/columns of P_(N) are added to P_(C). In this caseweighting factors {1/4, 1/8} are used for P_(N) and weighting factors{3/4, 7/8} are used for P_(C). For P_(N) generated based on motionvectors of vertically (horizontally) neighbouring sub-block, samples inthe same row (column) of P_(N) are added to P_(C) with a same weightingfactor.

In the JEM, for a CU with size less than or equal to 256 luma samples, aCU level flag is signalled to indicate whether OBMC is applied or notfor the current CU. For the CUs with size larger than 256 luma samplesor not coded with AMVP mode, OBMC is applied by default. At the encoder,when OBMC is applied for a CU, its impact is taken into account duringthe motion estimation stage. The prediction signal formed by OBMC usingmotion information of the top neighbouring block and the leftneighbouring block is used to compensate the top and left boundaries ofthe original signal of the current CU, and then the normal motionestimation process is applied.

2.2.5. Local Illumination Compensation

Local Illumination Compensation (LIC) is based on a linear model forillumination changes, using a scaling factor a and an offset b. And itis enabled or disabled adaptively for each inter-mode coded coding unit(CU).

When LIC applies for a CU, a least square error method is employed toderive the parameters a and b by using the neighbouring samples of thecurrent CU and their corresponding reference samples. More specifically,as illustrated in FIG. 19, the subsampled (2:1 subsampling) neighbouringsamples of the CU and the corresponding samples (identified by motioninformation of the current CU or sub-CU) in the reference picture areused. The IC parameters are derived and applied for each predictiondirection separately.

When a CU is coded with merge mode, the LIC flag is copied fromneighbouring blocks, in a way similar to motion information copy inmerge mode; otherwise, an LIC flag is signalled for the CU to indicatewhether LIC applies or not.

When LIC is enabled for a picture, additional CU level RD check isneeded to determine whether LIC is applied or not for a CU. When LIC isenabled for a CU, mean-removed sum of absolute difference (MR-SAD) andmean-removed sum of absolute Hadamard-transformed difference (MR-SATD)are used, instead of SAD and SATD, for integer pel motion search andfractional pel motion search, respectively.

To reduce the encoding complexity, the following encoding scheme isapplied in the JEM.

LIC is disabled for the entire picture when there is no obviousillumination change between a current picture and its referencepictures. To identify this situation, histograms of a current pictureand every reference picture of the current picture are calculated at theencoder. If the histogram difference between the current picture andevery reference picture of the current picture is smaller than a giventhreshold, LIC is disabled for the current picture; otherwise, LIC isenabled for the current picture.

2.2.6. Affine Motion Compensation Prediction

In HEVC, only translation motion model is applied for motioncompensation prediction (MCP). While in the real world, there are manykinds of motion, e.g. zoom in/out, rotation, perspective motions and heother irregular motions. In the JEM, a simplified affine transformmotion compensation prediction is applied. As shown in FIG. 20, theaffine motion field of the block is described by two control pointmotion vectors.

The motion vector field (MVF) of a block is described by the followingequation:

$\begin{matrix}\left\{ \begin{matrix}{v_{x} = {{\frac{\left( {v_{1\; x} - v_{0\; x}} \right)}{w}x} - {\frac{\left( {v_{1\; y} - v_{0\; y}} \right)}{w}y} + v_{0\; x}}} \\{v_{y} = {{\frac{\left( {v_{1y} - v_{0\; y}} \right)}{w}x} + {\frac{\left( {v_{1\; x} - v_{0\; x}} \right)}{w}y} + v_{0\; y}}}\end{matrix} \right. & \left( {1\text{-}a} \right)\end{matrix}$

For 6-parameter affine,

$\begin{matrix}\left\{ \begin{matrix}{v_{x} = {{\frac{v_{1\; x} - v_{0\; x}}{w}x} - {\frac{v_{2\; x} - v_{0\; x}}{w}y} + v_{0\; x}}} \\{v_{y} = {{\frac{v_{1y} - v_{0\; y}}{w}x} + {\frac{v_{2\; y} - v_{0\; y}}{w}y} + v_{0\; y}}}\end{matrix} \right. & \left( {1\text{-}b} \right)\end{matrix}$

Where (v_(0x), v_(0y)) is motion vector of the top-left corner controlpoint, and (v_(1x), v_(1y)) is motion vector of the top-right cornercontrol point and (v_(2x), v_(2y)) is motion vector of the bottom-leftcorner control point, (x, y) represents the coordinate of arepresentative point relative to the top-left sample within currentblock. In VTM, the representative point is defined to be the centerposition of a sub-block, e.g., when the coordinate of the left-topcorner of a sub-block relative to the top-left sample within currentblock is (xs, ys), the coordinate of the representative point is definedto be (xs+2, ys+2).

In order to further simplify the motion compensation prediction,sub-block based affine transform prediction is applied. The sub-blocksize M×N is derived as in Equation 2, where MvPre is the motion vectorfraction accuracy ( 1/16 in JEM), (v_(2x), v_(2y)) is motion vector ofthe bottom-left control point, calculated according to Equation 1.

$\begin{matrix}\left\{ \begin{matrix}{M = {{clip}\; 3\ \left( {4,w,\ \frac{w \times {MvPre}}{\max\left( {{{abs}\left( {v_{1\; x} - v_{0\; x}} \right)}_{j}{{abs}\left( {v_{1y} - v_{0y}} \right)}} \right)}} \right)}} \\{N = {{clip}\; 3\ \left( {4,h,\ \frac{h \times {MvPre}}{\max\left( {{{abs}\left( {v_{2\; x} - v_{0\; x}} \right)}_{j}{{abs}\left( {v_{2y} - v_{0y}} \right)}} \right)}} \right)}}\end{matrix} \right. & (2)\end{matrix}$

After derived by Equation 2, M and N should be adjusted downward ifnecessary to make it a divisor of w and h, respectively.

To derive motion vector of each M×N sub-block, the motion vector of thecenter sample of each sub-block, as shown in FIG. 21, is calculatedaccording to Equation 1, and rounded to 1/16 fraction accuracy. Then themotion compensation interpolation filters mentioned elsewhere herein areapplied to generate the prediction of each sub-block with derived motionvector.

After MCP, the high accuracy motion vector of each sub-block is roundedand saved as the same accuracy as the normal motion vector.

2.2.6.1. AF_INTER Mode

In the JEM, there are two affine motion modes: AF_INTER mode andAF_MERGE mode. For CUs with both width and height larger than 8,AF_INTER mode can be applied. An affine flag in CU level is signalled inthe bitstream to indicate whether AF_INTER mode is used. In this mode, acandidate list with motion vector pair {(v₀, v₁)|v₀ {v_(A), v_(B), v_(C)}, v₁ {v_(D),v_(E)}} is constructed using the neighbour blocks. As shownin FIG. 23, v₀ is selected from the motion vectors of the block A, B orC. The motion vector from the neighbour block is scaled according to thereference list and the relationship among the POC of the reference forthe neighbour block, the POC of the reference for the current CU and thePOC of the current CU. And the approach to select v₁ from the neighbourblock D and E is similar. If the number of candidate list is smallerthan 2, the list is padded by the motion vector pair composed byduplicating each of the AMVP candidates. When the candidate list islarger than 2, the candidates are firstly sorted according to theconsistency of the neighbouring motion vectors (similarity of the twomotion vectors in a pair candidate) and only the first two candidatesare kept. An RD cost check is used to determine which motion vector paircandidate is selected as the control point motion vector prediction(CPMVP) of the current CU. And an index indicating the position of theCPMVP in the candidate list is signalled in the bitstream. After theCPMVP of the current affine CU is determined, affine motion estimationis applied and the control point motion vector (CPMV) is found. Then thedifference of the CPMV and the CPMVP is signalled in the bitstream.

In AF_INTER mode, when 4/6 parameter affine mode is used, 2/3 controlpoints are required, and therefore 2/3 MVD needs to be coded for thesecontrol points, as shown in FIG. 22. It is proposed to derive the MV asfollows, i.e., mvd₁ and mvd₂ are predicted from mvd₀.

mv ₀ =mv ₀ +mvd ₀

mv ₁ =mv ₁ +mvd ₁ +mvd ₀

mv ₂ =mv ₂ +mvd ₂ +mvd ₀

Wherein mv _(i), mvd₁ and mv₁ are the predicted motion vector, motionvector difference and motion vector of the top-left pixel (i=0),top-right pixel (i=1) or left-bottom pixel (i=2) respectively, as shownin FIG. 22 (b). Please note that the addition of two motion vectors(e.g., mvA(xA, yA) and mvB(xB, yB)) is equal to summation of twocomponents separately, that is, newMV=mvA+mvB and the two components ofnewMV is set to (xA+xB) and (yA+yB), respectively.

2.2.6.2. AF_MERGE Mode

When a CU is applied in AF_MERGE mode, it gets the first block codedwith affine mode from the valid neighbour reconstructed blocks. And theselection order for the candidate block is from left, above, aboveright, left bottom to above left as shown in FIG. 24 a. If the neighbourleft bottom block A is coded in affine mode as shown in FIG. 24 b, themotion vectors v₂, v₃ and v₄ of the top left corner, above right cornerand left bottom corner of the CU which contains the block A are derived.And the motion vector v₀ of the top left corner on the current CU iscalculated according to v₂, v₃ and v₄. Secondly, the motion vector v₁ ofthe above right of the current CU is calculated.

After the CPMV of the current CU v₀ and v₁ are derived, according to thesimplified affine motion model Equation 1, the MVF of the current CU isgenerated. In order to identify whether the current CU is coded withAF_MERGE mode, an affine flag is signalled in the bitstream when thereis at least one neighbour block is coded in affine mode.

An affine merge candidate list is constructed with following steps:

Insert Inherited Affine Candidates

Inherited affine candidate means that the candidate is derived from theaffine motion model of its valid neighbor affine coded block. In thecommon base, as shown in FIG. 25, the scan order for the candidatepositions is: A1, B1, B0, A0 and B2.

After a candidate is derived, full pruning process is performed to checkwhether same candidate has been inserted into the list. If a samecandidate exists, the derived candidate is discarded.

Insert Constructed Affine Candidates

If the number of candidates in affine merge candidate list is less thanMaxNumAffineCand (set to 5 in this contribution), constructed affinecandidates are inserted into the candidate list. Constructed affinecandidate means the candidate is constructed by combining the neighbormotion information of each control point.

The motion information for the control points is derived firstly fromthe specified spatial neighbors and temporal neighbor shown in FIG. 25.CPk (k=1, 2, 3, 4) represents the k-th control point. A0, A1, A2, B0,B1, B2 and B3 are spatial positions for predicting CPk (k=1, 2, 3); T istemporal position for predicting CP4.

The coordinates of CP1, CP2, CP3 and CP4 is (0, 0), (W, 0), (H, 0) and(W, H), respectively, where W and H are the width and height of currentblock.

The motion information of each control point is obtained according tothe following priority order:

For CP1, the checking priority is B2->B3->A2. B2 is used if it isavailable. Otherwise, if B2 is available, B3 is used. If both B2 and B3are unavailable, A2 is used. If all the three candidates areunavailable, the motion information of CP1 cannot be obtained.

For CP2, the checking priority is B1->B0.

For CP3, the checking priority is A1->A0.

For CP4, T is used.

Secondly, the combinations of controls points are used to construct anaffine merge candidate.

Motion information of three control points are needed to construct a6-parameter affine candidate. The three control points can be selectedfrom one of the following four combinations ({CP1, CP2, CP4}, {CP1, CP2,CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). Combinations {CP1, CP2, CP3},{CP2, CP3, CP4}, {CP1, CP3, CP4} will be converted to a 6-parametermotion model represented by top-left, top-right and bottom-left controlpoints.

Motion information of two control points are needed to construct a4-parameter affine candidate. The two control points can be selectedfrom one of the following six combinations ({CP1, CP4}, {CP2, CP3},{CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}). Combinations {CP1,CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4} will be convertedto a 4-parameter motion model represented by top-left and top-rightcontrol points.

The combinations of constructed affine candidates are inserted into tocandidate list as following order:

{CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4},{CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

For reference list X (X being 0 or 1) of a combination, the referenceindex with highest usage ratio in the control points is selected as thereference index of list X, and motion vectors point to differencereference picture will be scaled.

After a candidate is derived, full pruning process is performed to checkwhether same candidate has been inserted into the list. If a samecandidate exists, the derived candidate is discarded.

Padding with zero motion vectors

If the number of candidates in affine merge candidate list is less than5, zero motion vectors with zero reference indices are insert into thecandidate list, until the list is full.

2.2.7. Bi-Directional Optical Flow

Bi-directional Optical flow (BIO) is sample-wise motion refinement whichis performed on top of block-wise motion compensation for bi-prediction.The sample-level motion refinement doesn't use signalling.

Let I^((k)) be the luma value from reference k (k=0, 1) after blockmotion compensation, and ∂I^((k))/∂x, ∂I^((k))/∂y are horizontal andvertical components of the I^((k)) gradient, respectively. Assuming theoptical flow is valid, the motion vector field (v_(x), v_(y)) is givenby an equation

∂I ^((k)) /∂t+v _(x) ∂I ^((k)) /∂x+v _(y) ∂I ^((k)) /∂y=0.  (3)

Combining this optical flow equation with Hermite interpolation for themotion trajectory of each sample results in a unique third-orderpolynomial that matches both the function values I^((k)) and derivatives∂I^((k))/∂x, ∂I^((k))/∂y at the ends. The value of this polynomial att=0 is the BIO prediction:

pred_(BIO)=½·(I ⁽⁰⁾ +I ⁽¹⁾ +v _(x)/2·(τ₁ ∂I ⁽¹⁾ /∂x−τ ₀ ∂I ⁽⁰⁾ /∂x)+v_(y)/2·(τ₁ ∂I ⁽¹⁾ /∂y−τ ₀ ∂I ⁽⁰⁾ ∂y)).  (3)

Here, τ₀ and τ₁ denote the distances to the reference frames as shown ona FIG. 26. Distances τ₀ and τ₁ are calculated based on POC for Ref0 andRef1: τ₀=POC(current)−POC(Ref0), τ₁=POC(Ref1)−POC(current). If bothpredictions come from the same time direction (either both from the pastor both from the future) then the signs are different (i.e., τ₀·τ₁<0).In this case, BIO is applied only if the prediction is not from the sametime moment (i.e., τ₀≠τ₁), both referenced regions have non-zero motion(MVx₀, MVy₀, MVx₁, MVy₁≠0) and the block motion vectors are proportionalto the time distance (MVx₀/MVx₁=MVy₀/MVy₁=−τ₀/τ₁).

The motion vector field (v_(x), v_(y)) is determined by minimizing thedifference Δ between values in points A and B (intersection of motiontrajectory and reference frame planes on FIG. 9). Model uses only firstlinear term of a local Taylor expansion for Δ:

Δ=(I ⁽⁰⁾ −I ⁽¹⁾ ₀ +v _(x)(τ₁ ∂I ⁽¹⁾ /∂x+τ ₀ ∂I ⁽⁰⁾ /∂x)+v _(y)(τ₁ ∂I ⁽¹⁾/∂y+τ ₀ ∂I ⁽⁰⁾ /∂y))  (5)

All values in Equation 5 depend on the sample location (i′,j′), whichwas omitted from the notation so far. Assuming the motion is consistentin the local surrounding area, we minimize Δ inside the (2M+1)×(2M+1)square window Ω centered on the currently predicted point (i,j), where Mis equal to 2:

$\begin{matrix}{\left( {v_{x},v_{y}} \right) = {\underset{v_{x}v_{y}}{\arg\;\min}{\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}{\Delta^{2}\left\lbrack {i^{\prime},j} \right\rbrack}}}} & (6)\end{matrix}$

For this optimization problem, the JEM uses a simplified approach makingfirst a minimization in the vertical direction and then in thehorizontal direction. This results in

$\begin{matrix}{v_{x} = {{\left( {s_{1} + r} \right) > {{m?{clip}}\; 3\left( {{- {thBIO}},{thBIO},{- \frac{s_{3}}{\left( {s_{1} + \tau} \right)}}} \right)}}:0}} & (7) \\{{v_{y} = {{\left( {s_{5} + r} \right) > {{m?{clip}}\; 3\left( {{- {thBIO}},{{thBIO} - \frac{s_{6} - {v_{x}{s_{2}/2}}}{\left( {s_{5} + \tau} \right)}}} \right)}}:0}}{{where},{{\left( {{s_{1} = {\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}\left( {{\tau_{1}\;{{\partial I^{(1)}}/{\partial x}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial x}}}} \right)^{2}}};} \right)^{2}s_{3}} = {\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}\left( {{I^{(1)} - {I^{(0)}\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial x}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial x}}}} \right\}}};} \right.}}}} & (8) \\{{{s_{2} = {\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}{\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial x}}} + {\tau_{0}{{\partial I^{(0)}}/\partial_{X}}}} \right)\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial y}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial y}}}} \right\}}}};}{{s_{5} = {\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial y}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial y}}}} \right)^{2}}};{s_{6} = {\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}{\left( {I^{(1)} - I^{(0)}} \right)\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial y}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial y}}}} \right)}}}}} & (9)\end{matrix}$

In order to avoid division by zero or a very small value, regularizationparameters r and m are introduced in Equations 7 and 8.

r=500·4^(d−8)  (10)

m=700·4^(d−8)  (11)

Here d is bit depth of the video samples.

In order to keep the memory access for BIO the same as for regularbi-predictive motion compensation, all prediction and gradients values,I^((k)), ∂I^((k))/∂x, ∂I^((k))/∂y, are calculated only for positionsinside the current block. In Equation 9, (2M+1)×(2M+1) square window Ωcentered in currently predicted point on a boundary of predicted blockneeds to accesses positions outside of the block (as shown in FIG. 27(a)). In the JEM, values of I^((k)), ∂I^((k))/∂x, ∂I^((k))/∂y outside ofthe block are set to be equal to the nearest available value inside theblock. For example, this can be implemented as padding, as shown in FIG.27 (b).

With BIO, it's possible that the motion field can be refined for eachsample. To reduce the computational complexity, a block-based design ofBIO is used in the JEM. The motion refinement is calculated based on 4×4block. In the block-based BIO, the values of s_(n) in Equation 9 of allsamples in a 4×4 block are aggregated, and then the aggregated values ofs_(n) in are used to derived BIO motion vectors offset for the 4×4block. More specifically, the following formula is used for block-basedBIO derivation:

$\begin{matrix}{{{s_{1,b_{k}} = {\sum\limits_{{({x,y})} \in b_{k}}{\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in {\Omega{({x,y})}}}\left( {\tau_{1}{{\partial I^{(1)}}/{\partial_{X}{+ \tau_{0}}}}{{\partial I^{(0)}}/{\partial x}}} \right)^{2}}}};{s_{3,b_{k}} = {\sum\limits_{{({x,y})} \in b_{k}}{\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}{\left( {I^{(1)} - I^{(0)}} \right)\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial x}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial x}}}} \right)}}}};}{{s_{2,b_{k}} = {\sum\limits_{{({x,y})} \in b_{k}}{\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}{\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial x}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial x}}}} \right)\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial y}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial y}}}} \right)}}}};}{{s_{5,b_{k}} = {\sum\limits_{{({x,y})} \in b_{k}}{\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}\left( {{\tau_{1}{{\partial I^{(1)}}/\phi}} + {\tau_{0}{{\partial I^{(0)}}/{\partial y}}}} \right)^{2}}}};{s_{6,b_{k}} = {\sum\limits_{{({x,y})} \in b_{k}}{\sum\limits_{{\lbrack{i^{\prime},j}\rbrack} \in \Omega}{\left( {I^{(1)} - I^{(0)}} \right)\left( {{\tau_{1}{{\partial I^{(1)}}/{\partial y}}} + {\tau_{0}{{\partial I^{(0)}}/{\partial y}}}} \right)}}}}}} & (12)\end{matrix}$

where b_(k) denotes the set of samples belonging to the k-th 4×4 blockof the predicted block. s_(n) in Equations 7 and 8 are replaced by((s_(n,bk))>>4) to derive the associated motion vector offsets.

In some cases, MV regiment of BIO might be unreliable due to noise orirregular motion. Therefore, in BIO, the magnitude of MV regiment isclipped to a threshold value thBIO. The threshold value is determinedbased on whether the reference pictures of the current picture are allfrom one direction. If all the reference pictures of the current pictureare from one direction, the value of the threshold is set to12×2^(14−d); otherwise, it is set to 12×2^(13−d).

Gradients for BIO are calculated at the same time with motioncompensation interpolation using operations consistent with HEVC motioncompensation process (2D separable FIR). The input for this 2D separableFIR is the same reference frame sample as for motion compensationprocess and fractional position (fracX, fracY) according to thefractional part of block motion vector. In case of horizontal gradient∂I/∂x signal first interpolated vertically using BIOfilterScorresponding to the fractional position fracY with de-scaling shiftd−8, then gradient filter BIOfilterG is applied in horizontal directioncorresponding to the fractional position fracX with de-scaling shift by18-d. In case of vertical gradient ∂I/∂y first gradient filter isapplied vertically using BIOfilterG corresponding to the fractionalposition fracY with de-scaling shift d−8, then signal displacement isperformed using BIOfilterS in horizontal direction corresponding to thefractional position fracX with de-scaling shift by 18-d. The length ofinterpolation filter for gradients calculation BIOfilterG and signaldisplacement BIOfilterF is shorter (6-tap) in order to maintainreasonable complexity. Table 3 shows the filters used for gradientscalculation for different fractional positions of block motion vector inBIO. Table 4 shows the interpolation filters used for prediction signalgeneration in BIG.

TABLE 1 Filters for gradients calculation in BIO FractionalInterpolation filter for pel position gradient(BIOfilterG) 0 {8, −39,−3, 46, −17, 5} 1/16 {8, −32, −13, 50, −18, 5} ⅛ {7, −27, −20, 54, −19,5} 3/16 {6, −21, −29, 57, −18, 5} ¼ {4, −17, −36, 60, −15, 4} 5/16 {3,−9, −44, 61, −15, 4} ⅜ {1, −4, −48, 61, −13, 3} 7/16 {0, 1, −54, 60, −9,2} ½ {−1, 4, −57, 57, −4, 1}

TABLE 2 Interpolation filters for prediction signal generation in BIOFractional Interpolation filter for prediction pel positionsignal(BIOfilterS) 0 {0, 0, 64, 0, 0, 0} 1/16 {1, −3, 64, 4, −2, 0} ⅛{1, −6, 62, 9, −3, 1} 3/16 {2, −8, 60, 14, −5, 1} ¼ {2, −9, 57, 19, −7,2} 5/16 {3, −10, 53, 24, −8, 2} ⅜ {3, −11, 50, 29, −9, 2} 7/16 {3, −11,44, 35, −10, 3} ½ {3, −10, 35, 44, −11, 3}

In the JEM, BIG is applied to all bi-predicted blocks when the twopredictions are from different reference pictures. When LIC is enabledfor a CU, BIG is disabled.

In the JEM, OBMC is applied for a block after normal MC process. Toreduce the computational complexity, BIG is not applied during the OBMCprocess. This means that BIG is only applied in the MC process for ablock when using its own MV and is not applied in the MC process whenthe MV of a neighboring block is used during the GBMC process.

2.2.8. Decoder-Side Motion Vector Refinement

In bi-prediction operation, for the prediction of one block region, twoprediction blocks, formed using a motion vector (MV) of list0 and a MVof list1, respectively, are combined to form a single prediction signal.In the decoder-side motion vector refinement (DMVR) method, the twomotion vectors of the bi-prediction are further refined by a bilateraltemplate matching process. The bilateral template matching applied inthe decoder to perform a distortion-based search between a bilateraltemplate and the reconstruction samples in the reference pictures inorder to obtain a refined MV without transmission of additional motioninformation.

In DMVR, a bilateral template is generated as the weighted combination(i.e. average) of the two prediction blocks, from the initial MV0 oflist0 and MV1 of list1, respectively, as shown in FIG. 28. The templatematching operation consists of calculating cost measures between thegenerated template and the sample region (around the initial predictionblock) in the reference picture. For each of the two reference pictures,the MV that yields the minimum template cost is considered as theupdated MV of that list to replace the original one. In the JEM, nine MVcandidates are searched for each list. The nine MV candidates includethe original MV and 8 surrounding MVs with one luma sample offset to theoriginal MV in either the horizontal or vertical direction, or both.Finally, the two new MVs, i.e., MV0′ and MV1′ as shown in FIG. 28, areused for generating the final bi-prediction results. A sum of absolutedifferences (SAD) is used as the cost measure. Please note that whencalculating the cost of a prediction block generated by one surroundingMV, the rounded MV (to integer pel) is actually used to obtain theprediction block instead of the real MV.

DMVR is applied for the merge mode of bi-prediction with one MV from areference picture in the past and another from a reference picture inthe future, without the transmission of additional syntax elements. Inthe JEM, when LIC, affine motion, FRUC, or sub-CU merge candidate isenabled for a CU, DMVR is not applied.

The LUT-based motion vector prediction based on the disclosedtechnology, which may enhance both existing and future video codingstandards, is elucidated in the following examples described for variousimplementations. Because the LUTs allow the encoding/decoding process tobe performed based on historical data (e.g., the blocks that have beenprocessed), the LUT-based motion vector prediction can also be referredto as History-based Motion Vector Prediction (HMVP) method. In theLUT-based motion vector prediction method, one or multiple tables withmotion information from previously coded blocks are maintained duringthe encoding/decoding process. These motion candidates stored in theLUTs are named HMVP candidates. During the encoding/decoding of oneblock, the associated motion information in LUTs may be added to themotion candidate lists (e.g., merge/AMVP candidate lists), and afterencoding/decoding one block, LUTs may be updated. The updated LUTs arethen used to code the subsequent blocks. That is, the updating of motioncandidates in the LUTs are based on the encoding/decoding order ofblocks. The examples below should be considered as examples to explaingeneral concepts. These examples should not be interpreted in a narrowway. Furthermore, these examples can be combined in any manner.

3. Examples of Problems Solved by Embodiments

In the design of triangle partition, one block could be split to twopartitions. To save memory bandwidth due to motion compensation, it isrequired that the two partitions shall be uni-predicted. During themotion compensation process, bi-prediction is used for the diagonal edgeand uni-prediction is used for all other remaining parts. Ifbi-prediction is allowed for each partition, then samples located at thediagonal edge will have four MVs with two of them from one partition andtwo of them from another partition. Such a design has the followingproblems:

Only spatial neighboring blocks and temporal blocks are checked in themerge list construction process.

History-based motion vector prediction technology is disallowed fortriangle partition.

How to handle triangular partition mode with intra block copy mode isunknown.

4. Examples of Embodiments

The proposed techniques could be applied to any ofnon-square/non-rectangular partitions, e.g., geometry partitions. In thefollowing descriptions, we use ‘triangular partition mode’ to representone as an example of the non-square/non-rectangular partition mode(TPM). It should be noted other kinds of partitions may be alsoapplicable.

The detailed techniques below should be considered as examples toexplain general concepts. These techniques should not be interpreted ina narrow way. Furthermore, these inventions can be combined in anymanner. Denote a block size as W×H.

-   -   1. Instead of always applying triangular prediction mode with        the merge mode, it is proposed to enable triangular prediction        mode for intra-coded blocks, or non-merge inter-coded blocks        -   a. In one example, the two partitions are predicted with            different intra prediction modes.        -   b. Alternatively, furthermore, the prediction values of            samples along edge boundaries may be further filtered.        -   c. In one example, the motion information (such as MV) of            the two triangular partitions cannot predict each other;            -   i. Alternatively, the motion information (such as MV) of                one triangular partition can be used to predict the                other partition.    -   2. It is proposed that when one block is split into two geometry        partitions, one partition may be coded with intra mode and the        other may be coded with inter mode.        -   a. In one example, for the inter-coded partition,            bi-prediction may be also applied.        -   b. In one example, such method may be signaled when current            block is coded with merge mode, i.e., for the inter-coded            partition, motion information signaled with an merge index.        -   c. Alternatively, furthermore, the prediction values of            samples along edge boundaries may be further filtered.        -   d. In one example, only a subset of intra prediction modes            may be applied.            -   i. The subsets may be different for the two partitions.            -   ii. The subsets may be dependent on the position of a                partition.            -   iii. The subsets may be dependent on the block size                and/or block shape.    -   3. As disclosed in Bullet 1 and Bullet 2, the merge or non-merge        inter-coded block can use the current picture as the reference        picture.    -   4. It is proposed that when coding one block with triangular        partition mode, inter/intra coded information of non-adjacent        spatial blocks may be also treated as predictors for coding        current block.        -   a. In one example, the motion information of non-adjacent            spatial blocks may be utilized.        -   b. In one example, intra prediction mode of non-adjacent            spatial blocks may be utilized.        -   c. Alternatively, coded information of temporal blocks may            be further utilized for coding one block with TPM.    -   5. It is proposed to add motion candidates derived from HMVP        candidates in the merge list construction process for TPM coded        blocks wherein a HMVP (History based motion vector prediction)        candidate is the motion information inherited or derived from        previously coded blocks.

Table Maintenance for HMVP Candidate Storage

-   -   a. The maintenance of HMVP candidates and/or updating of tables        for storing and/or HMVP candidates may be same as HMVP for        normal motion vector.    -   b. In one example, the same table may be maintained for HMVP        candidates storage which may be utilized for both non-TPM coded        blocks and TPM coded blocks.        -   i. In one example, motion information used by TPM are not            put into the HMVP storage. HMVP table(s) are not updated            after encoding/decoding a TPM coded block.    -   c. In one example, separate tables may be maintained to store        the HMVP candidates for coding TPM coded blocks.        -   i. Tables may be maintained for HMVP candidates storage with            motion information purely from TPM coded blocks.        -   ii. Two tables may be maintained to store uni- and            bi-predicted HMVP candidates, respectively.        -   iii. Two tables may be maintained to store motion            information of the first and second partitions,            respectively.        -   iv. Two tables may be maintained to store motion information            of List 0 and List 1 motion information, respectively.        -   v. Alternatively, three tables may be maintained to store            uni-predicted from List 0, uni-predicted from L1 and            bi-predicted HMVP candidates, respectively.    -   d. After coding with TPM coded block, tables for HMVP candidates        storage may not be updated.        -   i. Alternatively, one/multiple tables for HMVP candidates            storage may be updated with the motion information of the            first partition.        -   ii. Alternatively, one/multiple tables for HMVP candidates            storage may be updated with the motion information of the            second partition.        -   iii. Alternatively, one/multiple tables for HMVP candidates            storage may be updated with the motion information from both            partitions, by adding two HMVP candidates        -   iv. Alternatively, one/multiple tables for HMVP candidates            storage may be updated with the motion information from both            partitions, by adding one HMVP candidate (e.g., when the two            partitions are predicted from two reference picture lists).        -   v. Whether to add one or two motion candidates to the HMVP            table may depend on whether the two partitions are predicted            from the same reference picture lists and/or from same            reference picture.        -   vi. Whether to add motion information of the first partition            or the second partition to the HMVP table may depend on POC            difference between the reference pictures and the current            picture.    -   e. When separate tables are kept for coding TPM- and non-TPM        coded blocks, the updating of tables may be invoked based on the        mode of one block.        -   i. In one example, after decoding a TPM coded block, the            motion information may be used to update the table for            coding TPM-coded blocks.        -   ii. In one example, after decoding a non-TPM coded block,            the motion information may be used to update the table for            coding non-TPM-coded blocks.        -   iii. Alternatively, furthermore, after decoding a non-TPM            coded block, the motion information may be used to update            the table for coding TPM-coded blocks.

Usage of HMVP Candidates in TPM Coded Blocks

-   -   f. A HMVP candidate may be directly added to the merge list for        TPM-coded blocks.        -   i. Alternatively, one HMVP candidate may be utilized to            derive two motion candidates, e.g., one is uni-prediction            with List 0 motion information and the other is            uni-prediction with List 1 motion information of the HMVP            candidate.        -   ii. Pruning may be applied when inserting a motion candidate            derived/inherited from a HMVP candidate with other motion            candidates derived/inherited from spatial or temporal            blocks.        -   iii. Pruning may be applied when inserting a motion            candidate derived/inherited from a HMVP candidate with a            motion candidate derived/inherited from another HMVP            candidate.        -   iv. Pruning may be applied when inserting a motion candidate            derived/inherited from a HMVP candidate with another motion            candidate derived/inherited from the same HMVP candidate.    -   g. In one example, one or multiple motion candidates derived        from one or multiple HMVP candidates may be added after motion        candidates derived from spatial and/or temporal blocks such as        block 1-7 in FIG. 14. In this case, the HMVP candidate is        treated in the same way as the other spatial/temporal merge        candidates.    -   h. Motion candidates inherited/derived from HMVP candidates may        be added to the merge list right after or before a specific step        in the merge list construction process which relies on motion        information of spatial/temporal blocks.        -   i. Alternatively, motion candidates inherited/derived from            HMVP candidates may be added to the merge list right after            or before several pre-defined steps.        -   ii. In one example, one or multiple HMVP candidates with            uni-prediction may be added right after all originally            uni-predicted candidates with current design.        -   iii. In one example, one or multiple HMVP candidates with            uni-prediction from L0 or bi-prediction may be added right            after all Truncated List0-predicted candidates.            Alternatively, furthermore, if a HMVP candidate is with            bi-prediction, only List 0 motion information may be kept.        -   iv. In one example, one or multiple HMVP candidates with            uni-prediction from L1 or bi-prediction may be added right            after all Truncated List1-predicted candidates.            Alternatively, furthermore, if a HMVP candidate is with            bi-prediction, only List 1 motion information may be kept.        -   v. In one example, one or multiple HMVP candidates may be            added right after all averaged uni-prediction from List 0 or            List 1 motion candidates.        -   vi. In one example, one or multiple HMVP candidates may be            added right before all averaged uni-prediction from List 0            or List 1 motion candidates.    -   i. Motion candidates inherited/derived from HMVP candidates may        be added to the merge list in an interleaved way with motion        candidates inherited/derived from motion information of        spatial/temporal blocks.    -   j. In one example, motion candidates derived from List 0 and        List 1 of each HMVP candidate may be added in order before        motion candidates derived from List 0 and List 1 of another HMVP        candidate.    -   k. In one example, motion candidates derived from List 0 of a        first set of HMVP candidates may be added in order before those        derived from List 1 of a second set of HMVP candidate.        -   i. In one example, the first set and the second set are the            same set, i.e., all available HMVP candidates.        -   ii. Alternatively, the first set and the second set could be            different, such as the first set of HMVP candidates include            HMVP candidates with uni-prediction from List 0 and            bi-prediction; while the second set of HMVP candidates            include HMVP candidates with uni-prediction from List 1 and            bi-prediction.        -   iii. The number of HMVP candidates associated with the first            and the second set may be different.    -   l. HMVP candidates of list LX may be first scaled to list L(1−X)        and then used for list L(1−X).        -   i. The scaled candidates may be inserted after all other            candidates.        -   ii. The scaled candidates may be inserted after all other            candidates except the temporal candidates.    -   m. The checking order of HMVP candidates to be used to derive        motion candidates to be added to the TPM merge list may be        dependent on the indices of the HMVP candidates.        -   i. Alternatively, it may be dependent on the prediction            directions.        -   ii. Alternatively, it may be dependent on the motion            information.    -   6. How many HMVP to be checked in TPM merge list construction        process may be pre-defined, e.g., 5.        -   a. The number of HMVP candidate to be checked in TPM merge            list construction process may depend on block size/block            shape/how many candidates are available before checking HMVP            candidates.        -   b. The number of HMVP candidate to be checked in TPM merge            list construction process may be signaled in            SPS/VPS/PPS/picture header/slice header/tile group            header/CTU rows/CTU/group of CTUs.        -   c. When partial of available HMVP candidates are to be            checked in TPM merge list construction process, the            selection of HMVP candidates may depend on the prediction            direction/MV information/reference picture index/POC            distances of reference picture and current picture of a            candidate and/or candidate index.    -   7. Whether to and how to apply HMVP on a TPM coded block may        depend on the block size (e.g., width and/or height, ratio of        width and height) or block shape (e.g., square or non-square).    -   8. The proposed methods could also be applicable to other kinds        of motion candidate lists (such as AMVP list) for geometry        partitions.    -   9. The HMVP table size (i.e., maximum number of history motion        candidates could be stored) may depend on a function of one or        multiple motion candidate list sizes.        -   a. In one example, the HMVP table size may depend on the            regular merge list size, the TPM merge list size, the IBC            merge list size, the regular inter AMVP list size, the            regular inter IBC list size etc. al.        -   b. In one example, the HMVP table size may depend on those            motion candidate lists which access the same HMVP table.        -   c. In one example, the function may be the function            Max(inputs) which returns the maximum value among several            inputs.        -   d. In one example, the HMVP table size may be defined as            Max(regular merge list size Minus K0, TPM merge list size            Minus K1, IBC merge list size Minus K2).            -   i. In one example, K0=K1=K2=1.    -   10. The HMVP table size (i.e., maximum number of history motion        candidates could be stored) may depend on a function of one or        multiple HMVP candidate numbers (denoted by numHMVP) that may be        added to motion candidate lists.        -   a. In one example, it may depend on the number of HMVP            candidates that may be added to the motion candidate lists            (e.g., the regular merge list, the TPM merge list, the IBC            merge list, the regular inter AMVP list, the regular inter            IBC list etc. al).        -   b. In one example, the function may be the function            Max(inputs) which returns the maximum value among several            inputs.        -   c. In one example, the HMVP table size may be defined as            Max(numHMVP for regular merge list Minus K0, numHMVP for            regular AMVP list Minus K1).            -   ii. In one example, K0=1 and K1=0.        -   d. In one example, the HMVP table size may be defined as            Max(numHMVP for regular merge list Minus K0, numHMVP for TPM            merge list Minus K1, numHMVP for IBC merge list size Minus            K2).            -   iii. In one example, K0=K1=K2=1.    -   11. The HMVP table size for one coding mode may be different        from another coding mode, and it may be dependent on the motion        candidate list size.        -   a. In one example, the HMVP table size for the regular merge            and TPM merge lists may be dependent on the regular merge            list size.            -   iv. In one example, the HMVP table size may be equal to                regular merge list size minus K0, e.g. K0=1.        -   b. In one example, the HMVP table size for the regular merge            and TPM merge lists may be dependent on the regular merge            list size and the TPM merge list size.            -   v. In one example, it may be defined as Max(regular                merge list size Minus K0, TPM merge list size Minus K1).            -   vi. In one example, the HVMP table for the normal                inter-AMVP follows that used for the regular merge and                TPM merge lists.        -   c. In one example, the HMVP table size for the IBC-coded            blocks depend on that for the regular merge list and/or the            TPM merge list.            -   vii. For example, the HMVP table size for the IBC-coded                blocks is equal to that for the regular merge list                and/or the TPM merge list.            -   viii. Alternatively, the HMVP table size for the                IBC-coded merge/AVMP blocks may depend on the size of                IBC merge/AMVP candidate list size.        -   d. Indications of the HMVP table sizes may be signaled in            the bitstream.            -   ix. When the signaled size is equal to 0, HMVP                candidates are not inserted to any motion candidate                lists.            -   x. In one example, HMVP table size is not allowed to be                0.            -   xi. Alternatively, HMVP table size minus K0 is signaled,                e.g. K0=1.    -   12. Indications of number of allowed HMVP candidates that may be        added to motion candidate lists may be signaled in the        bitstream.        -   a. In one example, for each mode (e.g., IBC or non-IBC;            merge no AMVP), an indication of number of allowed HMVP            candidates may be signaled independently.        -   b. Alternatively, the indications of numbers of allowed HMVP            candidates may be signaled in a predictive way.        -   c. Alternatively, furthermore, the HMVP table sizes may            depend on the allowed number of HMVP candidates.        -   d. For example, the number of allowed HMVP candidates that            may be added to motion candidate lists is not allowed to be            0.        -   e. Alternatively, the number of allowed HMVP candidates that            may be added to motion candidate lists minus K0 may be            signaled, e.g. K0=1.        -   f. Alternatively, predictive coding of the number of allowed            HMVP candidates that may be added to a motion candidate list            (denoted by numHMVP) and the number of maximum motion            candidate in the motion candidate list (denoted by            numMotionList) may be applied.            -   xii. In one example, the difference between                numMotionList and numHMVP may be coded.            -   xiii. In one example, the difference between                (numMotionList-K0) and numHMVP may be coded, e.g., K0=1.    -   13. Number of allowed HMVP candidates that may be added to        motion candidate lists may be dependent on the motion candidate        list sizes.        -   a. In one example, it may be derived from the corresponding            motion candidate list size, such as (list size−K0), e.g.,            K0=0 or 1.        -   b. In one example, it may be derived from the size of the            corresponding HMVP table that is used by the motion            candidate list, such as (table size−K0), e.g., K0=0 or 1.    -   14. A conformance bitstream shall satisfy that the number of        allowed HMVP candidates that may be added to a motion candidate        list is no larger than the number of maximum motion candidate in        the motion candidate list.        -   a. Alternatively, a conformance bitstream shall satisfy that            the number of allowed HMVP candidates that may be added to a            motion candidate list is no larger than (the number of            maximum motion candidate in the motion candidate list minus            K0), e.g., K0=1.

FIG. 29 is a block diagram of a video processing apparatus 2900. Theapparatus 2900 may be used to implement one or more of the methodsdescribed herein. The apparatus 2900 may be embodied in a smartphone,tablet, computer, Internet of Things (IoT) receiver, and so on. Theapparatus 2900 may include one or more processors 2902, one or morememories 2904 and video processing hardware 2906. The processor(s) 2902may be configured to implement one or more methods described in thepresent document. The memory (memories) 2904 may be used for storingdata and code used for implementing the methods and techniques describedherein. The video processing hardware 2906 may be used to implement, inhardware circuitry, some techniques described in the present document.

FIG. 31 is a flowchart for a method 3100 of processing a video. Themethod 3100 includes performing a determination (3105) that a firstvideo block is intra-coded or non-merge inter-coded, determining (3110)a first prediction portion and a second prediction portion for the firstvideo block based on the determination that the first video block isintra-coded or non-merge inter-coded, wherein one or both of the firstprediction portion or the second prediction portion are non-rectangularand non-square portions of the first video block, and performing (3115)further processing of the first video block using the first predictionportion and the second prediction portion.

FIG. 32 is a flowchart for a method 3200 of processing a video. Themethod 3200 includes performing a determination (3205) that a firstvideo block is intra-coded or non-merge inter-coded, determining (3210)a first prediction portion and a second prediction portion for the firstvideo block, wherein one or both of the first prediction portion or thesecond prediction portion are non-rectangular and non-square portions ofthe first video block, and performing (3215) further processing of thefirst video block using the first prediction portion and the secondprediction portion, wherein at least one prediction portion is merge ornon-merge inter coded using a current image as a reference image.

FIG. 33 is a flowchart for a method 3300 of processing a video. Themethod 3300 includes performing a determination (3305) that a firstvideo block is intra-coded or non-merge inter-coded, determining (3310)a first prediction portion and a second prediction portion for the firstvideo block, wherein one or both of the first prediction portion or thesecond prediction portion are non-rectangular and non-square portions ofthe first video block, and performing (3315) further processing of thefirst video block using the first prediction portion and the secondprediction portion, wherein performing further processing of the firstvideo block using the first prediction portion and the second predictionportion is based on inter or intra coded information of non-adjacentspatial blocks in relation to the first video block.

FIG. 34 is a flowchart for a method 3400 of processing a video. Themethod 3400 includes performing a determination (3405) that a firstvideo block is coded with triangular portion mode (TPM) using atriangular prediction portion of the first video block and that a secondvideo block is coded using non-TPM using a non-triangular predictionportion of the second video block, performing (3410) further processingof the first video block and the second video block using stored HMVPcandidates, and storing (3415) HMVP candidates associated with the firstvideo block and the second video block.

FIG. 35 is a flowchart for a method 3500 of processing a video. Themethod 3500 includes performing a determination (3505) that a firstvideo block includes prediction portions that are non-rectangular andnon-square portions of the first video block, identifying (3510) an HMVPcandidate, adding (3515) one or more motion candidates derived from theHMVP candidate to a merge list associated with video blocks that includeprediction portions that are non-rectangular and non-square, andperforming (3520) further processing of the first video block using themerge list.

With reference to methods 3100, 3200, 3300, 3400, and 3500, someexamples of determining a candidate for encoding and their use aredescribed in Section 4 of the present document. For example, asdescribed in Section 4, video blocks can be processed using predictionportions that are non-square and non-rectangular.

With reference to methods 3100, 3200, 3300, 3400, and 3500, a videoblock may be encoded in the video bitstream in which bit efficiency maybe achieved by using a bitstream generation rule related to motioninformation prediction.

The methods can include determining image information of the firstprediction portion using a first intra prediction mode; and determiningimage information of the second prediction portion using a second intraprediction mode, the first intra prediction mode being different thanthe second intra prediction mode.

The methods can include filtering prediction values related to the firstprediction portion and the second prediction portion along an edgeboundary between the first prediction portion and the second predictionportion.

The methods can include determining, by the processor, first motioninformation of the first prediction portion; and determining, by theprocessor, second motion information of the second prediction portion,wherein the first motion information is determined without using thesecond motion information, and the second motion information isdetermined without using the first motion information.

The methods can include determining, by the processor, first motioninformation of the first prediction portion; and determining, by theprocessor, second motion information of the second prediction portion,wherein the second motion information is determined using the firstmotion information.

The methods can include processing the first prediction portion withintra mode coding; and processing the second prediction portion withinter mode coding.

The methods can include wherein processing the second prediction portionincludes applying bi-prediction.

The methods can include wherein processing the second prediction portionincludes determining motion information signaled with a merge index.

The methods can include filtering prediction values related to the firstprediction portion and the second prediction portion along an edgeboundary between the first prediction portion and the second predictionportion.

The methods can include wherein a subset of intra mode coding is used.

The methods can include wherein the first prediction portion and thesecond prediction portion are associated with different subsets of intramode coding.

The methods can include wherein the first prediction portion and thesecond prediction portion are associated with different subsets of intramode coding based on positions of the first prediction portion and thesecond prediction portion.

The methods can include wherein the first prediction portion and thesecond prediction portion are associated with different subsets of intramode coding based on one or more of: size of the first video block, orshape of the first video block.

The methods can include wherein the inter or intra coded information ofnon-adjacent spatial video blocks includes motion information of thenon-adjacent spatial video blocks.

The methods can include wherein an intra prediction mode of thenon-adjacent spatial video blocks is used.

The methods can include wherein coded information of temporal blocks isused.

The methods can include adding history based motion vector prediction(HMVP) candidates in a motion vector prediction candidate list, whereinthe HMVP candidates includes motion information based on previouslycoded video blocks; and decoding the first video block based on the HMVPcandidates.

The methods can include wherein the HMVP candidates are stored in atable, and wherein motion information used by TPM is not stored in thetable.

The methods can include wherein the table is not updated after encodingor decoding the first video block using TPM.

The methods can include wherein the HMVP candidates are stored in afirst table and a second table, the first table storing HMVP candidatesfor TPM, and the second table storing HMVP candidates for non-TPM.

The methods can include wherein one of the first table or the secondtable stores uni-predicted HMVP candidates, and the other storesbi-predicted HMVP candidates.

The methods can include wherein one of the first table or the secondtable stores motion information of the first partition portion, and theother stores motion information of the second partition portion.

The methods can include wherein one of the first table or the secondtable stores motion information of List 0, and the other stores motioninformation of List 1.

The methods can include wherein the first table stores uni-predictedHMVP candidates from List 0, the second table stores uni-predicted HMVPcandidates from List 1, and a third table stores bi-predicted HMVPcandidates.

The methods can include wherein tables for HMVP candidate storage is notupdated based on one or both of the first prediction portion or thesecond prediction portion being non-rectangular and non-square geometryportions of the first video block.

The methods can include updating one or more of the first table or thesecond table for HMVP candidates storage with motion information of thefirst partition portion.

The methods can include updating one or more of the first table or thesecond table for HMVP candidates storage with motion information of thesecond partition portion.

The methods can include updating one or more of the first table or thesecond table for HMVP candidates storage with motion information of thefirst partition portion and motion information of the second partitionportion.

The methods can include determining that the first partition portion andthe second partition portion are predicted from two reference picturelists; and updating one or more tables for HMVP candidates storage withmotion information of one HMVP candidate based on the determination thatthe first partition portion and the second partition portion arepredicted from two reference picture lists.

The methods can include determining that the first partition portion andthe second partition portion are predicted using a same referencepicture list or a same reference picture, wherein the tables are updatedbased on the determination that the first partition portion and thesecond partition portion are predicted using the same reference picturelist or the same reference picture.

The methods can include determining a picture order count (POC)difference between a reference picture and a current picture, whereinmotion information of the first partition portion or the secondpartition portion are added to the one or more tables based on the POCdifference.

The methods can include determining characteristics regarding coding ofthe first video block; and updating one or both of the first table orthe second table based on the characteristics regarding coding of thefirst video block.

The methods can include wherein motion information is used to update atable associated with TPM video blocks after decoding of a TPM videoblock.

The methods can include wherein motion information is used to update atable associated with non-TPM video blocks after decoding of a non-TPMvideo block.

The methods can include wherein motion information is used to update atable associated with TPM video blocks after decoding of a non-TPM videoblock.

The methods can include determining a first motion candidate and asecond motion candidate from the HMVP candidate, one of the first mergecandidate or the second merge candidate being uni-prediction with List 0motion information of the HMVP candidate, and the other beinguni-prediction with List 1 motion information of the HMVP candidate.

The methods can include pruning based on an insertion of a motioncandidate derived from the HMVP candidate with other motion candidatesthat are based on spatial or temporal video blocks.

The methods can include pruning based on an insertion of a motioncandidate derived from the HMVP candidate with other motion candidatesthat are based on other HMVP candidates.

The methods can include pruning based on an insertion of a motioncandidate derived from the HMVP candidate with other motion candidatesthat are based on the HMVP candidate.

The methods can include adding the motion candidates derived from theHMVP candidate associated with the first video block to a merge listafter motion candidates derived from one or both of spatial or temporalvideo blocks.

The methods can include adding a motion candidate derived from the HMVPcandidate to the merge list after or before a merge list constructionprocess based on motion information of spatial or temporal video blocks.

The methods can include adding a motion candidate derived from the HMVPcandidate to the merge list after or before a pre-defined step.

The methods can include adding the motion candidates derived from theHMVP candidate with uni-prediction after other originally uni-predictedcandidates.

The methods can include adding the motion candidates derived from theHMVP candidate with uni-prediction from List 0 or bi-prediction aftertruncated List 0 predicted candidates.

The methods can include adding the motion candidates derived from theHMVP candidate with uni-prediction from List 1 or bi-prediction aftertruncated List 1 predicted candidates.

The methods can include adding the motion candidates derived from theHMVP candidate after averaged uni-prediction from List 0 or List 1motion candidates.

The methods can include adding the motion candidates derived from theHMVP candidate before averaged uni-prediction from List 0 or List 1motion candidates.

The methods can include wherein motion candidates derived from HMVPcandidates are added to the merge list in accordance with interleavingwith motion candidates that are based on motion information of spatialor temporal video blocks in relation to the first video block.

The methods can include wherein motion candidates derived from List 0and List 1 of the HMVP candidate is added in order before motioncandidates derived from List 0 and List 1 of another HMVP candidate.

The methods can include herein motion candidates derived from List 0 ofa first set of HMVP candidates are added in order before motioncandidates derived from List 1 of a second set of HMVP candidates.

The methods can include wherein the first set and the second set are thesame, wherein the first set and the second set include all availableHMVP candidates.

The methods can include wherein the first set and the second set aredifferent, the first set including HMVP candidates with uni-predictionfrom List 0 and bi-prediction, the second set including HMVP candidateswith uni-prediction from List 1 and bi-prediction.

The methods can include wherein the first set includes a first number ofHMVP candidates, the second set includes a second number of HMVPcandidates, the first number and the second number being different.

The methods can include wherein a scaled motion candidate may be derivedfrom a HMVP candidate by scaling its motion vector of list LX to listL(1−X) and the scaled motion candidate is assigned with the scaledmotion vector and list L(1−X).

The methods can include wherein the scaled motion candidates derivedfrom HMVP candidates are inserted after other motion candidates derivedfrom HMVP candidates.

The methods can include wherein the scaled motion candidates derivedfrom HMVP candidates are inserted before temporal candidates.

The methods can include wherein a checking order of HMVP candidates tobe added to the merge list is based on indices of the HMVP candidates.

The methods can include wherein a checking order of HMVP candidates tobe added to the merge list is based on prediction directions.

The methods can include wherein a checking order of HMVP candidates tobe added to the merge list is based on motion information.

The methods can include wherein a number of HMVP candidates to bechecked in a TPM merge list construction process is pre-defined.

The methods can include wherein a number of HMVP candidates to bechecked in a TPM merge list construction process is based on a blocksize of the first video block, a block shape of the first video block,or a number of candidates available before checking HMVP candidates.

The methods can include wherein a number of HMVP candidates to bechecked in a TPM merge list construction process is signaled in a videoparameter set (VPS), a sequence parameter set (SPS), a picture parameterset (PPS), a picture header, a tile group header, a slice header, agroup of rows of coding tree units (CTUs), a CTU, or a group of CTUs.

The methods can include wherein selection of the HMVP candidates isbased on a prediction direction, a motion vector (MV) information, areference picture index, a POC distances of reference picture andcurrent picture of one or both of: a candidate, or a candidate index.

The methods can include wherein application of HMVP in association withthe first video block is based on a block size of the first video blockor a block shape of the first video block.

The methods can include wherein the non-rectangular and non-squareportions are triangular.

It will be appreciated that the disclosed techniques may be embodied invideo encoders or decoders to improve compression efficiency when thecoding units being compressed have shaped that are significantlydifferent than the traditional square shaped blocks or rectangularblocks that are half-square shaped. For example, new coding tools thatuse long or tall coding units such as 4×32 or 32×4 sized units maybenefit from the disclosed techniques.

FIG. 36 is a flowchart for a method 3600 of processing a video. Themethod 3600 includes determining (3605), during a conversion between acurrent block of visual media data and a corresponding codedrepresentation of the visual media data, multiple sub-portions for thecurrent block; determining (3610) intra prediction information of themultiple sub-portions; and performing (3615) performing the conversionof the current block using the intra prediction information of themultiple sub-portions; and wherein the current block is intra-coded, andat least one of the multiple sub-portions is a non-rectangular andnon-square sub-portion. It should be noted that the sub-portion mayequivalent to the prediction portion above.

In some implementations, additional modifications can be performed tomethod 3600. For example, determining intra prediction information ofthe multiple sub-portions comprises: determining a first intraprediction information of a first sub-portion in the multiplesub-portions using a first intra prediction mode; determining a secondintra prediction information of a second sub-portion in the multiplesub-portions using a second intra prediction mode; wherein the firstintra prediction mode is different from the second intra predictionmode. The first intra prediction information is determined by using afirst subset of intra prediction modes, and the second intra predictioninformation is determined by using second subset of intra predictionmodes; wherein the first subset of intra prediction modes is differentfrom the second subset of intra prediction modes. The subset of intraprediction modes is based on at least one of a position, a size and ashape of the first sub-portion and/or second sub-portion. Determiningintra prediction information of the multiple sub-portions furthercomprises: filtering prediction values of samples along an edge boundaryof the at least one of the multiple sub-portions. First intra predictioninformation of a first sub-portion of the multiple sub-portions isdetermined without using the second intra prediction information of asecond sub-portion of the multiple sub-portions, and the second intraprediction information of the second sub-portion is determined withoutusing the first intra prediction information of the first sub-portion.First intra prediction information of a first sub-portion of themultiple sub-portions is determined using second intra predictioninformation of a second sub-portion of the multiple sub-portions, and/orthe second intra prediction information of the second sub-portion isdetermined using the first intra prediction information of the firstsub-portion.

FIG. 37 is a flowchart for a method 3700 of processing a video. Themethod 3700 includes determining (3705), during a conversion between acurrent block of visual media data and a corresponding codedrepresentation of the visual media data, multiple sub-portions for thecurrent block; determining (3710) motion information of the multiplesub-portions; performing (3715) the conversion of the current blockusing the motion information of the multiple sub-portions; and whereinthe current block is non-merge inter-coded, and at least one of themultiple sub-portions is a non-rectangular and non-square sub-portion.

In some implementations, additional modifications can be performed tomethod 3700. For example, determining motion information of the multiplesub-portions comprises: determining a first motion information of afirst sub-portion in the multiple sub-portions using a first intraprediction mode; determining a second motion information of a secondsub-portion in the multiple sub-portions using a second intra predictionmode. Determining intra prediction of the multiple sub-portions furthercomprises: filtering prediction values of samples along an edge boundaryof the at least one of the multiple sub-portions. The first motioninformation of the first sub-portion is determined without using thesecond motion information of the second sub-portion, and the secondmotion information of the second sub-portion is determined without usingthe first motion information of the first sub-portion. The first motioninformation of the first sub-portion is determined using the secondmotion information of the second prediction, and/or the second motioninformation of the second sub-portion is determined using the firstmotion information of the first sub-portion.

FIG. 38 is a flowchart for a method 3800 of processing a video. Themethod 3800 includes performing (3805) a conversion between a currentblock of visual media data and a corresponding coded representation ofthe visual media data, wherein the current block is partitioned intomultiple sub-portions according to a splitting pattern in which a firstsub-portion has a non-rectangular, non-square shape; processing (3810) afirst sub-portion with intra coding mode; and processing (3815) a secondsub-portion with inter coding mode.

In some implementations, additional modifications can be performed tomethod 3800. For example, the second sub-portion is processed byapplying bi-prediction. Signaling motion information with an merge indexin response to the current block being coded with merge mode.Determining motion information of the first predication portion using asubset of intra prediction mode. The subset of intra prediction modes isbased on at least one of a position of the first predication portion, asize of a first predication portion and a shape of a first predicationportion.

FIG. 39 is a flowchart for a method 3900 of processing a video. Themethod 3900 includes performing (3905) a conversion between a currentblock of visual media data and a corresponding coded representation ofthe visual media data, wherein the current block is partitioned intomultiple sub-portions according to a splitting pattern in which a firstsub-portion has a non-rectangular, non-square shape; wherein the atleast one of the multiple sub-portions is merge or non-merge inter codedand uses a current picture as a reference picture.

FIG. 40 is a flowchart for a method 4000 of processing a video. Themethod 4000 includes performing (4005) a conversion between a currentblock of visual media data and a corresponding coded representation ofthe visual media data, wherein the current block is partitioned intomultiple sub-portions according to a splitting pattern in which a firstprediction partition has a non-rectangular, non-square shape; andperforming (4010) the conversion using inter or intra coded informationof one or more non-adjacent spatial blocks.

In some implementations, additional modifications can be performed tomethod 4000. For example, the inter or intra coded information of one ormore non-adjacent spatial blocks comprises motion information of the oneor more non-adjacent spatial blocks. Performing the conversion of thecurrent block using inter or intra coded information of one or morenon-adjacent spatial blocks comprises: performing the conversion of thecurrent block using intra prediction mode of the one or morenon-adjacent spatial blocks. Coded information of temporal blocks isused.

FIG. 41 is a flowchart for a method 4100 of processing a video. Themethod 4100 includes determining (4105), during a conversion between afirst block of visual media data and a corresponding codedrepresentation of the visual media data, the first block being codedwith geometry partition mode; determining (4110) based on at least onetable storing history based motion vector prediction (HMVP) candidateswhich include motion information based on previously coded blocks,motion information of at least one sub-portion of the first block;performing (4115) the conversion of the first block using the determinedmotion information.

In some implementations, additional modifications can be performed tomethod 4100. For example, the first block being coded with geometrypartition mode comprises: splitting the first block to multiplesub-portions; wherein at least one of the multiple sub-portions is anon-rectangular and non-square portion. Determining, during a conversionbetween a second block of the visual media data and the correspondingcoded representation of the visual media data, motion information of thesecond block based on the same table that is used for the first block,and the second block is not using the geometry partition mode. Themotion information used by the first block is refrained from beingstored in the at least one table. The at least one table is not updatedafter the conversion of the first block. The at least one table includesmultiple tables maintained to store the HMVP candidates for blocks withthe geometry partition mode. For the at least one table, the stored HMVPcandidates only have motion information according to that used by blockswith the geometry partition mode. The at least one table includes twotables maintained to store uni-predicted HMVP candidates andbi-predicted HMVP candidates, respectively. The at least one tableincludes two tables maintained to store motion information of a firstsub-portion and a second sub-portion of the first block, respectively.The at least one table includes two tables maintained to store motioninformation of List 0 and List 1, respectively. The at least one tableincludes three tables maintained to store uni-predicted HMVP candidatesfrom List 0, uni-predicted HMVP candidates from List 1, and bi-predictedHMVP candidates, respectively. The at least one table is updated withthe motion information of part of sub-portions of the first block. Theat least one table is updated with the motion information of a firstsub-portion and the motion information of a second sub-portion, thefirst sub-portion and the second sub-portion being sub-portion s of thefirst block. The at least one table is updated with the motioninformation of a first sub-portion and the motion information of asecond sub-portion by adding one HMVP candidate. Whether to update theat least one table by adding one or two motions candidates depends onwhether the first sub-portion and the second sub-portion are using asame reference picture list or a same reference picture. Whether toupdate the at least one table by adding motion information of the firstsub-portion or the second sub-portion depends on a picture order count(POC) difference between a reference picture and a current picture.After processing the block with the non-rectangular and non-squareportion, the motion information of the block with the non-rectangularand non-square portion is used to update the table maintained to storethe HMVP candidates for block with the non-rectangular and non-squareportion. After processing the block without the non-rectangular andnon-square portion, the motion information of the block without thenon-rectangular and non-square portion is used to update the tablemaintained to store the HMVP candidates for block without thenon-rectangular and non-square portion. After processing the blockwithout the non-rectangular and non-square portion, the motioninformation of the block without the non-rectangular and non-squareportion is used to update the table maintained to store the motionvector prediction (HMVP) candidates for block with the non-rectangularand non-square portion. The geometry partition mode comprises atriangular partitioning mode. The method is applicable to other kinds ofmotion candidate lists.

FIG. 42 is a flowchart for a method 4200 of processing a video. Themethod 4200 includes determining (4205), during a conversion between afirst block of visual media data and a corresponding codedrepresentation of the visual media data, the first block being codedwith geometry partition mode; determining (4210) motion information ofat least one sub-portion of the first block; performing (4215)performing the conversion of the first block using the motioninformation of the at least one sub-portion; wherein determining motioninformation of at least one sub-portion comprises using at least onehistory based motion vector prediction (HMVP) candidate which includesmotion information based on a previously coded block to construct amotion candidate list and determining the motion information from themotion candidate list.

In some implementations, additional modifications can be performed tomethod 4200. For example, the first block being coded with geometrypartition mode comprises: splitting the first block to multiplesub-portions; wherein at least one of the multiple sub-portions is anon-rectangular and non-square portion. The at least one HMVP candidateis directly added to the motion candidate list. The using at least oneHMVP candidate which includes motion information based on a previouslycoded block to construct a motion candidate list comprises: using oneHMVP candidate to determine a first motion candidate and a second motioncandidate, one of the first motion candidate and the second motioncandidate being uni-prediction with List 0 motion information of theHMVP candidate, and the other being uni-prediction with List 1 motioninformation of the HMVP candidate. Performing a pruning in response toinserting a motion candidate derived from the HMVP candidate with othermotion candidates that are based on spatial or temporal video blocks.Performing a pruning in response to inserting a motion candidate derivedfrom the HMVP candidate with other motion candidates that are based onother HMVP candidates. Performing a pruning in response to inserting amotion candidate derived from the HMVP candidate with other motioncandidates that are based on the HMVP candidate. The using at least oneHMVP candidate which includes motion information based on a previouslycoded block to construct a motion candidate list comprises: adding atleast one motion candidate derived from the HMVP candidate to the motioncandidate list after motion candidates derived from one or both ofspatial or temporal blocks. The using at least one HMVP candidate whichincludes motion information based on a previously coded block toconstruct a motion candidate list comprises: adding at least one motioncandidate derived from the HMVP candidate to the motion candidate listafter or before a pre-defined step in a motion candidate listconstruction process based on motion information of spatial or temporalblocks. The using at least one HMVP candidate which includes motioninformation based on a previously coded block to construct a motioncandidate list comprises: adding at least one motion candidate derivedfrom the HMVP candidate to the motion candidate list after or beforemultiple pre-defined steps in a motion candidate list constructionprocess based on motion information of spatial or temporal blocks. Theusing at least one HMVP candidate which includes motion informationbased on a previously coded block to construct a motion candidate listcomprises: adding at least one motion candidate derived from the HMVPcandidate with uni-prediction to the motion candidate list after alloriginally uni-predicted candidates. The using at least one HMVPcandidate which includes motion information based on a previously codedblock to construct a motion candidate list comprises: adding at leastone motion candidate derived from the HMVP candidate with uni-predictionfrom List 0 or bi-prediction to the motion candidate list aftertruncated List 0 predicted candidates. The using at least one HMVPcandidate which includes motion information based on a previously codedblock to construct a motion candidate list comprises: adding at leastone motion candidate derived from the HMVP candidate with uni-predictionfrom List 1 or bi-prediction to the motion candidate list aftertruncated List 1 predicted candidates. The using at least one HMVPcandidate which includes motion information based on a previously codedblock to construct a motion candidate list comprises: adding at leastone motion candidate derived from the at least one HMVP candidate to themotion candidate list after averaged uni-prediction from List 0 or List1 motion candidates. The using at least one HMVP candidate whichincludes motion information based on a previously coded block toconstruct a motion candidate list comprises: adding at least one motioncandidates derived from the at least one HMVP candidate to the motioncandidate list before averaged uni-prediction from List 0 or List 1motion candidates. The using at least one HMVP candidate which includesmotion information based on a previously coded block to construct amotion candidate list comprises: adding at least one motion candidatederived from the HMVP candidate to the motion candidate list inaccordance with interleaving with motion candidates that are based onmotion information of spatial or temporal blocks in relation to thecurrent block. The derived motion candidate from a HMVP candidate is setequal to the HMVP candidate. The using at least one HMVP candidate whichincludes motion information based on a previously coded block toconstruct a motion candidate list comprises: adding at least one motioncandidate derived from List 0 and List 1 of the HMVP candidate to themotion candidate list before motion candidates derived from List 0 andList 1 of another HMVP candidate. The using at least one HMVP candidatewhich includes motion information based on a previously coded block toconstruct a motion candidate list comprises: adding at least one motioncandidate derived from List 0 of a first set of HMVP candidates to themotion candidate list before the motion candidates derived from List 1of a second set of HMVP candidates. The first set and the second set arethe same. The first set and the second set are different, and the firstset includes HMVP candidates with uni-prediction from List 0 andbi-prediction, the second set includes HMVP candidates withuni-prediction from List 1 and bi-prediction. The first set includes afirst number of HMVP candidates, the second set includes a second numberof HMVP candidates, and the first number and the second number aredifferent. The using at least one HMVP candidate which includes motioninformation based on a previously coded block to construct a motioncandidate list comprises: scaling the HMVP candidates of list LX to listL(1−X) to generate the scaled HMVP candidates which are used for theL(1−X). The using at least one HMVP candidate which includes motioninformation based on a previously coded block to construct a motioncandidate list comprises: adding the scaled HMVP candidates after othermotion candidates derived from HMVP candidates. The using at least oneHMVP candidate which includes motion information based on a previouslycoded block to construct a motion candidate list comprises: adding thescaled HMVP candidates after other motion candidates derived from HMVPcandidates and before temporal candidates. A checking order of HMVPcandidates to be added to the motion candidate list is based on indicesof the HMVP candidates. A checking order of HMVP candidates to be addedto the motion candidate list is based on prediction directions. Achecking order of HMVP candidates to be added to the motion candidatelist is based on motion information. A number of HMVP candidates to bechecked in the motion candidate list construction process ispre-defined. A number of HMVP candidates to be checked in the motioncandidate list construction process is based on a block size of thecurrent block, a block shape of the current block, or a number ofcandidates available before checking HMVP candidates. A number of HMVPcandidates to be checked in the motion candidate list constructionprocess is signaled in a video parameter set (VPS), a sequence parameterset (SPS), a picture parameter set (PPS), a picture header, a tile groupheader, a slice header, a group of rows of coding tree units (CTUs), aCTU, or a group of CTUs. The HMVP candidates selected to be checked inthe motion candidate list are based on at least one of a predictiondirection, a motion vector (MV) information, a reference picture index,a POC distances of reference picture and current picture of a candidateand a candidate index. An application of HMVP to the current block isbased on a block size of the current block or a block shape of thecurrent block. The motion candidate list comprises a merge list. Themotion candidate list is applicable to other kinds of motion candidatelists exclude the merge candidate list.

Some features preferably implemented by some embodiments are nowdisclosed in clause-based format.

1. A method for processing video, comprising:

determining, during a conversion between a current block of visual mediadata and a corresponding coded representation of the visual media data,multiple sub-portions for the current block;

determining intra prediction information of the multiple sub-portions;

performing the conversion of the current block using the intraprediction information of the multiple sub-portions; and

wherein the current block is intra-coded, and at least one of themultiple sub-portions is a non-rectangular and non-square sub-portion.

2. The method of solution 1, wherein determining intra predictioninformation of the multiple sub-portions comprises:

determining a first intra prediction information of a first sub-portionin the multiple sub-portions using a first intra prediction mode;

determining a second intra prediction information of a secondsub-portion in the multiple sub-portions using a second intra predictionmode;

wherein the first intra prediction mode is different from the secondintra prediction mode.

3. The method of solution 2, wherein the first intra predictioninformation is determined by using a first subset of intra predictionmodes, and the second intra prediction information is determined byusing second subset of intra prediction modes;

wherein the first subset of intra prediction modes is different from thesecond subset of intra prediction modes.

4. The method of solution 3, wherein the subset of intra predictionmodes is based on at least one of a position, a size and a shape of thefirst sub-portion and/or second sub-portion.

5. The method of anyone of solutions 1 to 4, wherein determining intraprediction information of the multiple sub-portions further comprises:

filtering prediction values of samples along an edge boundary of the atleast one of the multiple sub-portions.

6. The method of anyone of solutions 1 to 5, wherein first intraprediction information of a first sub-portion of the multiplesub-portions is determined without using the second intra predictioninformation of a second sub-portion of the multiple sub-portions, andthe second intra prediction information of the second sub-portion isdetermined without using the first intra prediction information of thefirst sub-portion.

7. The method of anyone of solutions 1 to 5, wherein first intraprediction information of a first sub-portion of the multiplesub-portions is determined using second intra prediction information ofa second sub-portion of the multiple sub-portions, and/or the secondintra prediction information of the second sub-portion is determinedusing the first intra prediction information of the first sub-portion.

8. A method for processing video, comprising:

determining, during a conversion between a current block of visual mediadata and a corresponding coded representation of the visual media data,multiple sub-portions for the current block;

determining motion information of the multiple sub-portions;

performing the conversion of the current block using the motioninformation of the multiple sub-portions; and

wherein the current block is non-merge inter-coded, and at least one ofthe multiple sub-portions is a non-rectangular and non-squaresub-portion.

9. The method of solution 8, wherein determining motion information ofthe multiple sub-portions comprises:

determining a first motion information of a first sub-portion in themultiple sub-portions using a first intra prediction mode;

determining a second motion information of a second sub-portion in themultiple sub-portions using a second intra prediction mode.

10. The method of anyone of solutions 8-9, wherein determining intraprediction of the multiple sub-portions further comprises:

filtering prediction values of samples along an edge boundary of the atleast one of the multiple sub-portions.

11. The method of anyone of solutions 8-9, wherein the first motioninformation of the first sub-portion is determined without using thesecond motion information of the second sub-portion, and the secondmotion information of the second sub-portion is determined without usingthe first motion information of the first sub-portion.

12. The method of anyone of solutions 8-9, wherein the first motioninformation of the first sub-portion is determined using the secondmotion information of the second prediction, and/or the second motioninformation of the second sub-portion is determined using the firstmotion information of the first sub-portion.

13. A method of video processing, comprising:

performing a conversion between a current block of visual media data anda corresponding coded representation of the visual media data, whereinthe current block is partitioned into multiple sub-portions according toa splitting pattern in which a first sub-portion has a non-rectangular,non-square shape;

processing a first sub-portion with intra coding mode; and

processing a second sub-portion with inter coding mode.

14. The method of solution 13, wherein the second sub-portion isprocessed by applying bi-prediction.

15. The method of anyone of solutions 13-14, further comprises:

signaling motion information with an merge index in response to thecurrent block being coded with merge mode.

16. The method of anyone of solutions 13-15, further comprises:

determining motion information of the first predication portion using asubset of intra prediction mode.

17. The method of solution 16, wherein the subset of intra predictionmodes is based on at least one of a position of the first predicationportion, a size of a first predication portion and a shape of a firstpredication portion.

18. A method of video processing, comprising:

performing a conversion between a current block of visual media data anda corresponding coded representation of the visual media data, whereinthe current block is partitioned into multiple sub-portions according toa splitting pattern in which a first sub-portion has a non-rectangular,non-square shape;

wherein the at least one of the multiple sub-portions is merge ornon-merge inter coded and uses a current picture as a reference picture.

19. A method of video processing, comprising:

performing a conversion between a current block of visual media data anda corresponding coded representation of the visual media data, whereinthe current block is partitioned into multiple sub-portions according toa splitting pattern in which a first prediction partition has anon-rectangular, non-square shape;

and performing the conversion using inter or intra coded information ofone or more non-adjacent spatial blocks.

20. The method of solution 19, wherein the inter or intra codedinformation of one or more non-adjacent spatial blocks comprises motioninformation of the one or more non-adjacent spatial blocks.

21. The method of solution 19, wherein performing the conversion of thecurrent block using inter or intra coded information of one or morenon-adjacent spatial blocks comprises:

performing the conversion of the current block using intra predictionmode of the one or more non-adjacent spatial blocks.

22. The method of solution 19, wherein coded information of temporalblocks is used.

23. A video processing apparatus comprising a processor configured toimplement a method recited in anyone of solutions 1 to 22.

24. The apparatus of solution 23, wherein the apparatus is a videoencoder.

25. The apparatus of solution 23, wherein the apparatus is a videodecoder.

26. A computer readable recoding media on which a program comprisingcode is recorded, the program is for a processor to carry out a methodrecited in anyone of solutions 1 to 22.

Some features preferably implemented by some embodiments are nowdisclosed in clause-based format.

1. A method for processing video, comprising:

determining, during a conversion between a first block of visual mediadata and a corresponding coded representation of the visual media data,the first block being coded with geometry partition mode;

determining, based on at least one table storing history based motionvector prediction (HMVP) candidates which include motion informationbased on previously coded blocks, motion information of at least onesub-portion of the first block;

performing the conversion of the first block using the determined motioninformation.

2 A method of solution 1, wherein the first block being coded withgeometry partition mode comprises:

splitting the first block to multiple sub-portions;

wherein at least one of the multiple sub-portions is a non-rectangularand non-square portion.

3. The method of solution 1, further comprising:

determining, during a conversion between a second block of the visualmedia data and the corresponding coded representation of the visualmedia data, motion information of the second block based on the sametable that is used for the first block,

and the second block is not using the geometry partition mode.

4. The method of anyone of solutions 1-3, wherein the motion informationused by the first block is refrained from being stored in the at leastone table.

5. The method of anyone of solutions 1-4, wherein the at least one tableis not updated after the conversion of the first block.

6. The method of anyone of solutions 1-5, wherein the at least one tableincludes multiple tables maintained to store the HMVP candidates forblocks with the geometry partition mode.

7. The method of solution 6, wherein for the at least one table, thestored HMVP candidates only have motion information according to thatused by blocks with the geometry partition mode.

8. The method of anyone of solutions 1-7, wherein the at least one tableincludes two tables maintained to store uni-predicted HMVP candidatesand bi-predicted HMVP candidates, respectively.

9. The method of anyone of solutions 1-8, wherein the at least one tableincludes two tables maintained to store motion information of a firstsub-portion and a second sub-portion of the first block, respectively.

10. The method of anyone of solutions 1-9, wherein the at least onetable includes two tables maintained to store motion information of List0 and List 1, respectively.

11. The method of anyone of solutions 1-10, wherein the at least onetable includes three tables maintained to store uni-predicted HMVPcandidates from List 0, uni-predicted HMVP candidates from List 1, andbi-predicted HMVP candidates, respectively.

12. The method of anyone of solutions 1-11, wherein the at least onetable is updated with the motion information of part of sub-portions ofthe first block.

13. The method of anyone of solutions 1-12, wherein the at least onetable is updated with the motion information of a first sub-portion andthe motion information of a second sub-portion, the first sub-portionand the second sub-portion being sub-portion s of the first block.

14. The method of solution 13, wherein the at least one table is updatedwith the motion information of a first sub-portion and the motioninformation of a second sub-portion by adding one HMVP candidate.

15. The method of solution 14, wherein whether to update the at leastone table by adding one or two motions candidates depends on whether thefirst sub-portion and the second sub-portion are using a same referencepicture list or a same reference picture.

16. The method of solution 14, wherein whether to update the at leastone table by adding motion information of the first sub-portion or thesecond sub-portion depends on a picture order count (POC) differencebetween a reference picture and a current picture.

17. The method of any one of solutions 1 to 16, wherein the methodfurther comprises:

after processing the block with the non-rectangular and non-squareportion, the motion information of the block with the non-rectangularand non-square portion is used to update the table maintained to storethe HMVP candidates for block with the non-rectangular and non-squareportion.

18. The method of solution 1 to 16, the method further comprises:

after processing the block without the non-rectangular and non-squareportion, the motion information of the block without the non-rectangularand non-square portion is used to update the table maintained to storethe HMVP candidates for block without the non-rectangular and non-squareportion.

19. The method of solution 1 to 16, the method further comprises:

after processing the block without the non-rectangular and non-squareportion, the motion information of the block without the non-rectangularand non-square portion is used to update the table maintained to storethe motion vector prediction (HMVP) candidates for block with thenon-rectangular and non-square portion.

20. The method of anyone of solutions 1-19, wherein the geometrypartition mode comprises a triangular partitioning mode.

21. The method of anyone of solutions 1-20, wherein the method isapplicable to other kinds of motion candidate lists.

22. A video processing apparatus comprising a processor configured toimplement a method recited in anyone of solutions 1 to 21.

23. The apparatus of solution 22, wherein the apparatus is a videoencoder.

24. The apparatus of solution 22, wherein the apparatus is a videodecoder.

25. A computer readable recoding media on which a program comprisingcode is recorded, the program is for a processor to carry out a methodrecited in anyone of solutions 1 to 21.

Some features preferably implemented by some embodiments are nowdisclosed in clause-based format.

1. A method for processing video, comprising:

determining, during a conversion between a first block of visual mediadata and a corresponding coded representation of the visual media data,the first block being coded with geometry partition mode;

determining motion information of at least one sub-portion of the firstblock;

performing the conversion of the first block using the motioninformation of the at least one sub-portion;

wherein determining motion information of at least one sub-portioncomprises using at least one history based motion vector prediction(HMVP) candidate which includes motion information based on a previouslycoded block to construct a motion candidate list and determining themotion information from the motion candidate list.

2 The method of solution 1, wherein the first block being coded withgeometry partition mode comprises:

splitting the first block to multiple sub-portions;

wherein at least one of the multiple sub-portions is a non-rectangularand non-square portion.

3. The method of solution 1, wherein

the at least one HMVP candidate is directly added to the motioncandidate list.

4. The method of anyone of solutions 1-3, wherein the using at least oneHMVP candidate which includes motion information based on a previouslycoded block to construct a motion candidate list comprises:

using one HMVP candidate to determine a first motion candidate and asecond motion candidate, one of the first motion candidate and thesecond motion candidate being uni-prediction with List 0 motioninformation of the HMVP candidate, and the other being uni-predictionwith List 1 motion information of the HMVP candidate.

5. The method of anyone of solutions 1-4, further comprising:

performing a pruning in response to inserting a motion candidate derivedfrom the HMVP candidate with other motion candidates that are based onspatial or temporal video blocks.

6. The method of anyone of solutions 1-5, further comprising:

performing a pruning in response to inserting a motion candidate derivedfrom the HMVP candidate with other motion candidates that are based onother HMVP candidates.

7. The method of anyone of solutions 1-6, further comprising:

performing a pruning in response to inserting a motion candidate derivedfrom the HMVP candidate with other motion candidates that are based onthe HMVP candidate.

8. The method of anyone of solutions 1-7, wherein the using at least oneHMVP candidate which includes motion information based on a previouslycoded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidate tothe motion candidate list after motion candidates derived from one orboth of spatial or temporal blocks.

9. The method of anyone of solutions 1-8, wherein the using at least oneHMVP candidate which includes motion information based on a previouslycoded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidate tothe motion candidate list after or before a pre-defined step in a motioncandidate list construction process based on motion information ofspatial or temporal blocks.

10. The method of anyone of solutions 1-9, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidate tothe motion candidate list after or before multiple pre-defined steps ina motion candidate list construction process based on motion informationof spatial or temporal blocks.

11. The method of anyone of solutions 1-10, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidatewith uni-prediction to the motion candidate list after all originallyuni-predicted candidates.

12. The method of anyone of solutions 1-11, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidatewith uni-prediction from List 0 or bi-prediction to the motion candidatelist after truncated List 0 predicted candidates.

13. The method of anyone of solutions 1-12, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidatewith uni-prediction from List 1 or bi-prediction to the motion candidatelist after truncated List 1 predicted candidates.

14. The method of anyone of solutions 1-13, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the at least one HMVPcandidate to the motion candidate list after averaged uni-predictionfrom List 0 or List 1 motion candidates.

15. The method of anyone of solutions 1-14, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidates derived from the at least one HMVPcandidate to the motion candidate list before averaged uni-predictionfrom List 0 or List 1 motion candidates.

16. The method of anyone of solutions 1-15, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from the HMVP candidate tothe motion candidate list in accordance with interleaving with motioncandidates that are based on motion information of spatial or temporalblocks in relation to the current block.

17. The method of anyone of solutions 8-16, wherein

the derived motion candidate from a HMVP candidate is set equal to theHMVP candidate

18. The method of anyone of solutions 1-17, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from List 0 and List 1 ofthe HMVP candidate to the motion candidate list before motion candidatesderived from List 0 and List 1 of another HMVP candidate.

19. The method of anyone of solutions 1-18, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

adding at least one motion candidate derived from List 0 of a first setof HMVP candidates to the motion candidate list before the motioncandidates derived from List 1 of a second set of HMVP candidates.

20. The method of solution 19, wherein the first set and the second setare the same.

21. The method of solution 19, wherein the first set and the second setare different, and the first set includes HMVP candidates withuni-prediction from List 0 and bi-prediction, the second set includesHMVP candidates with uni-prediction from List 1 and bi-prediction.

22. The method of solution 19, wherein the first set includes a firstnumber of HMVP candidates, the second set includes a second number ofHMVP candidates, and the first number and the second number aredifferent.

23. The method of anyone of solutions 1-22, wherein the using at leastone HMVP candidate which includes motion information based on apreviously coded block to construct a motion candidate list comprises:

scaling the HMVP candidates of list LX to list L(1−X) to generate thescaled HMVP candidates which are used for the L(1−X).

24. The method of solution 23, wherein the using at least one HMVPcandidate which includes motion information based on a previously codedblock to construct a motion candidate list comprises:

adding the scaled HMVP candidates after other motion candidates derivedfrom HMVP candidates.

25. The method of solution 23, wherein the using at least one HMVPcandidate which includes motion information based on a previously codedblock to construct a motion candidate list comprises:

adding the scaled HMVP candidates after other motion candidates derivedfrom HMVP candidates and before temporal candidates.

26. The method of anyone of solutions 1-25, wherein a checking order ofHMVP candidates to be added to the motion candidate list is based onindices of the HMVP candidates.

27. The method of anyone of solutions 1-25, wherein a checking order ofHMVP candidates to be added to the motion candidate list is based onprediction directions.

28. The method of anyone of solutions 1-25, wherein a checking order ofHMVP candidates to be added to the motion candidate list is based onmotion information.

29. The method of anyone of solutions 1-28, wherein a number of HMVPcandidates to be checked in the motion candidate list constructionprocess is pre-defined.

30. The method of anyone of solutions 1-28, wherein a number of HMVPcandidates to be checked in the motion candidate list constructionprocess is based on a block size of the current block, a block shape ofthe current block, or a number of candidates available before checkingHMVP candidates.

31. The method of anyone of solutions 1-28, wherein a number of HMVPcandidates to be checked in the motion candidate list constructionprocess is signaled in a video parameter set (VPS), a sequence parameterset (SPS), a picture parameter set (PPS), a picture header, a tile groupheader, a slice header, a group of rows of coding tree units (CTUs), aCTU, or a group of CTUs.

32. The method of anyone of solutions 1-28, wherein the HMVP candidatesselected to be checked in the motion candidate list are based on atleast one of a prediction direction, a motion vector (MV) information, areference picture index, a POC distances of reference picture andcurrent picture of a candidate and a candidate index.

33. The method of anyone of solutions 1-28, wherein an application ofHMVP to the current block is based on a block size of the current blockor a block shape of the current block.

34. The method of anyone of solutions 1-33, wherein the motion candidatelist comprises a merge list.

35. The method of anyone of solutions 1-33, wherein the motion candidatelist is applicable to other kinds of motion candidate lists exclude themerge candidate list.

36. A video processing apparatus comprising a processor configured toimplement a method recited in anyone of solutions 1 to 35.

37. The apparatus of solution 36, wherein the apparatus is a videoencoder.

38. The apparatus of solution 36, wherein the apparatus is a videodecoder.

39. A computer readable recoding media on which a program comprisingcode is recorded, the program is for a processor to carry out a methodrecited in anyone of solutions 1 to 35.

The disclosed and other solutions, examples, embodiments, modules andthe functional operations described in this document can be implementedin digital electronic circuitry, or in computer software, firmware, orhardware, including the structures disclosed in this document and theirstructural equivalents, or in combinations of one or more of them. Thedisclosed and other embodiments can be implemented as one or morecomputer program products, i.e., one or more modules of computer programinstructions encoded on a computer readable medium for execution by, orto control the operation of, data processing apparatus. The computerreadable medium can be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more them. The term “data processing apparatus” encompassesall apparatus, devices, and machines for processing data, including byway of example a programmable processor, a computer, or multipleprocessors or computers. The apparatus can include, in addition tohardware, code that creates an execution environment for the computerprogram in question, e.g., code that constitutes processor firmware, aprotocol stack, a database management system, an operating system, or acombination of one or more of them. A propagated signal is anartificially generated signal, e.g., a machine-generated electrical,optical, or electromagnetic signal, that is generated to encodeinformation for transmission to suitable receiver apparatus.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this document can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random-access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Computer readable media suitable for storingcomputer program instructions and data include all forms of non-volatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto optical disks; and CD ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in, special purposelogic circuitry.

While this patent document contains many specifics, these should not beconstrued as limitations on the scope of any subject matter or of whatmay be claimed, but rather as descriptions of features that may bespecific to particular embodiments of particular techniques. Certainfeatures that are described in this patent document in the context ofseparate embodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

What is claimed is:
 1. A method for coding video data, comprising:determining, during a conversion between a first block of visual mediadata and a corresponding bitstream of the visual media data, the firstblock which is a coding unit being coded with a geometric partitioningmode which is a prediction mode for the coding unit; constructing, basedon a table storing one or multiple motion candidates which areassociated with motion information based on previously coded blocks, amotion candidate list which corresponds to a merge candidate list,wherein the table includes a history based motion vector prediction(HMVP) table; determining a geometric partitioning mode merge index forgeometric partitions of the first block being included in thecorresponding bitstream, wherein the geometric partitioning mode mergeindex and the motion candidate list are related to motion information ofthe first block; and performing the conversion of the first block usingthe motion information of the first block, wherein the table is notupdated after the conversion of the first block, and wherein the methodfurther comprises: determining, during a conversion between a secondblock of the visual media data and the corresponding bitstream of thevisual media data, motion information of the second block based on thesame table that is used for the first block, wherein the second block isnot coded using the geometric partitioning mode.
 2. The method of claim1, wherein the motion information used by the first block is refrainedfrom being stored in the table.
 3. The method of claim 1, whereinconstructing the motion candidate list further comprises constructingthe motion candidate list based on motion candidates derived fromspatial and/or temporal blocks.
 4. The method of claim 1, wherein theconstructing, based on the table storing one or multiple motioncandidates which are associated with motion information based onpreviously coded blocks, the motion candidate list which corresponds tothe merge candidate list comprising: adding at least one motioncandidate derived from the table for the first block.
 5. The method ofclaim 4, wherein the adding motion candidates derived from the table forthe first block, comprising: adding at least one motion candidatederived from the table to the motion candidate list after motioncandidates derived from one or both of spatial blocks or temporalblocks.
 6. The method of claim 4, wherein the derived at least onemotion candidate from the table is set equal to corresponding at leastone motion candidate in the table.
 7. The method of claim 4, wherein achecking order of motion candidates in the table to be added to themotion candidate list is based on indices of motion candidates in thetable.
 8. The method of claim 1, wherein the geometric partitioning modeincludes multiple partition schemes and at least one partition schemedivides the first block in to two geometric partitions, at least one ofwhich is non-square and non-rectangular.
 9. The method of claim 1,wherein the geometric partitioning mode comprises a triangularpartitioning mode.
 10. The method of claim 1, wherein the conversioncomprises decoding the first block from the corresponding bitstream. 11.The method of claim 1, wherein the conversion comprises encoding thefirst block into the corresponding bitstream.
 12. An apparatus forcoding video data comprising a processor and a non-transitory memorywith instructions thereon, wherein the instructions upon execution bythe processor, cause the processor to: determine, during a conversionbetween a first block of visual media data and a corresponding bitstreamof the visual media data, the first block which is a coding unit beingcoded with a geometric partitioning mode which is a prediction mode forthe coding unit; construct, based on a table storing one or multiplemotion candidates which are associated with motion information based onpreviously coded blocks, a motion candidate list which corresponds to amerge candidate list, wherein the table includes a history based motionvector prediction (HMVP) table; determine a geometric partitioning modemerge index for geometric partitions of the first block being includedin the corresponding bitstream, wherein the geometric partitioning modemerge index and the motion candidate list are related to motioninformation of the first block; and perform the conversion of the firstblock using the motion information of the first block, wherein the tableis not updated after the conversion of the first block, and wherein theprocessor is further caused to: determine, during a conversion between asecond block of the visual media data and the corresponding bitstream ofthe visual media data, motion information of the second block based onthe same table that is used for the first block, wherein the secondblock is not coded using the geometric partitioning mode.
 13. Theapparatus of claim 12, wherein the motion information used by the firstblock is refrained from being stored in the table.
 14. The apparatus ofclaim 12, wherein the processor is further caused to: construct themotion candidate list based on motion candidates derived from spatialand/or temporal blocks.
 15. A non-transitory computer-readable storagemedium storing instructions that cause a processor to: determine, duringa conversion between a first block of visual media data and acorresponding bitstream of the visual media data, the first block whichis a coding unit being coded with a geometric partitioning mode which isa prediction mode for the coding unit; construct, based on a tablestoring one or multiple motion candidates which are associated withmotion information based on previously coded blocks, a motion candidatelist which corresponds to a merge candidate list, wherein the tableincludes a history based motion vector prediction (HMVP) table;determine a geometric partitioning mode merge index for geometricpartitions of the first block being included in the correspondingbitstream, wherein the geometric partitioning mode merge index and themotion candidate list are related to motion information of the firstblock; and perform the conversion of the first block using the motioninformation of the first block, wherein the table is not updated afterthe conversion of the first block, and wherein the processor is furthercaused to: determine, during a conversion between a second block of thevisual media data and the corresponding bitstream of the visual mediadata, motion information of the second block based on the same tablethat is used for the first block, wherein the second block is not codedusing the geometric partitioning mode.
 16. The non-transitorycomputer-readable storage medium of claim 15, wherein the motioninformation used by the first block is refrained from being stored inthe table.
 17. The non-transitory computer-readable storage medium ofclaim 15, wherein the processor is further caused to: construct themotion candidate list based on motion candidates derived from spatialand/or temporal blocks.
 18. A non-transitory computer-readable recordingmedium storing a bitstream which is generated by a method performed by avideo processing apparatus, wherein the method comprises: determining,during generating a corresponding bitstream of visual media data from afirst block of the visual media data, the first block which is a codingunit being coded with a geometric partitioning mode which is aprediction mode for the coding unit; constructing, based on a tablestoring one or multiple motion candidates which are associated withmotion information based on previously coded blocks, a motion candidatelist which corresponds to a merge candidate list, wherein the tableincludes a history based motion vector prediction (HMVP) table;determining a geometric partitioning mode merge index for geometricpartitions of the first block being included in the correspondingbitstream, wherein the geometric partitioning mode merge index and themotion candidate list are related to motion information of the firstblock; and generating the corresponding bitstream from the first blockusing the motion information of the first block, wherein the table isnot updated after the generating the corresponding bitstream from thefirst block, and wherein the method further comprises: determiningmotion information of the second block based on the same table that isused for the first block, wherein the second block is not coded usingthe geometric partitioning mode.
 19. The non-transitorycomputer-readable recording medium of claim 18, wherein the motioninformation used by the first block is refrained from being stored inthe table.
 20. The non-transitory computer-readable recording medium ofclaim 18, wherein constructing the motion candidate list furthercomprises constructing the motion candidate list based on motioncandidates derived from spatial and/or temporal blocks.